recallapp/MobileCLIP-B-LT-OpenCLIP
By recallapp · ↓ 3 · ♥ 0 · Updated 2025-02-22
What's in this repo
This page lists the files in recallapp/MobileCLIP-B-LT-OpenCLIP, including text_model_batch.onnx, text_model_q8_batch.onnx, vision_model_batch.onnx and more.
- 4 ONNX models (.onnx files)
- 17 files, including 5 .json, 4 .onnx, 1 .md, 1 .png, 1 .txt, 1 .mjs
- Last updated 2025-02-22
ONNX files
| File | Size | Download |
|---|---|---|
| onnx/text_model_batch.onnx | — | Download |
| onnx/text_model_q8_batch.onnx | — | Download |
| onnx/vision_model_batch.onnx | — | Download |
| onnx/vision_model_q8_batch.onnx | — | Download |
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training
MobileCLIP was introduced in MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training (CVPR 2024), by Pavan Kumar Anasosalu Vasu, Hadi Pouransari, Fartash Faghri, Raviteja Vemulapalli, Oncel Tuzel.
This repository contains the MobileCLIP-B (LT) checkpoint for OpenCLIP.
Highlights
- Our smallest variant
MobileCLIP-S0obtains similar zero-shot performance as OpenAI's ViT-B/16 model while being 4.8x faster and 2.8x smaller. MobileCLIP-S2obtains better avg zero-shot performance than SigLIP's ViT-B/16 model while being 2.3x faster and 2.1x smaller, and trained with 3x less seen samples.MobileCLIP-B(LT) attains zero-shot ImageNet performance of 77.2% which is significantly better than recent works like DFN and SigLIP with similar architectures or even OpenAI's ViT-L/14@336.
Checkpoints
| Model | # Seen
Samples (B) | # Params (M)
(img + txt) | Latency (ms)
(img + txt) | IN-1k Zero-Shot
Top-1 Acc. (%) | Avg. Perf. (%)
on 38 datasets |
|:----------------------------------------------------------|:----------------------:|:-----------------------------:|:-----------------------------:|:-----------------------------------:|:----------------------------------:|
| MobileCLIP-S0 | 13 | 11.4 + 42.4 | 1.5 + 1.6 | 67.8 | 58.1 |
| MobileCLIP-S1 | 13 | 21.5 + 63.4 | 2.5 + 3.3 | 72.6 | 61.3 |
| MobileCLIP-S2 | 13 | 35.7 + 63.4 | 3.6 + 3.3 | 74.4 | 63.7 |
| MobileCLIP-B | 13 | 86.3 + 63.4 | 10.4 + 3.3 | 76.8 | 65.2 |
| MobileCLIP-B (LT) | 36 | 86.3 + 63.4 | 10.4 + 3.3 | 77.2 | 65.8 |
Model info
Created: 2025-01-12
Updated: 2025-02-22
View on Hugging Face