dragonSwing/nanosam

NanoSAM: Accelerated Segment Anything Model for Edge deployment

• GitHub
• Demo

Pretrained Models

NanoSAM performance on edge devices. Latency/throughput is measured on NVIDIA Jetson Xavier NX, and NVIDIA T4 GPU with TensorRT, fp16. Data transfer time is included.

Zero-Shot Instance Segmentation on COCO2017 validation dataset

| Image Encoder | mAPmask
50-95 | mIoU (all) | mIoU (large) | mIoU (medium) | mIoU (small) | | --------------- | :-------------------: | :--------: | :----------: | :-----------: | :----------: | | ResNet18 | - | 70.6 | 79.6 | 73.8 | 62.4 | | MobileSAM | - | 72.8 | 80.4 | 75.9 | 65.8 | | PPHGV2-B1 | 41.2 | 75.6 | 81.2 | 77.4 | 70.8 | | PPHGV2-B2 | 42.6 | 76.5 | 82.2 | 78.5 | 71.5 | | PPHGV2-B4 | 44.0 | 77.3 | 83.0 | 79.7 | 72.1 | | EfficientViT-L0 | 45.6 | 78.6 | 83.7 | 81.0 | 73.3 |

Usage

from nanosam.utils.predictor import Predictor

image_encoder_cfg = {
    "path": "data/sam_hgv2_b4_ln_nonorm_image_encoder.onnx",
    "name": "OnnxModel",
    "provider": "cpu",
    "normalize_input": False,
}
mask_decoder_cfg = {
    "path": "data/efficientvit_l0_mask_decoder.onnx",
    "name": "OnnxModel",
    "provider": "cpu",
}
predictor = Predictor(encoder_cfg, decoder_cfg)

image = PIL.Image.open("assets/dogs.jpg")

predictor.set_image(image)

mask, _, _ = predictor.predict(np.array([[x, y]]), np.array([1]))

The point labels may be