agoor97/onnx-models
By agoor97 · ↓ 0 · ♥ 0 · Updated 2025-04-28
What's in this repo
This page lists the files in agoor97/onnx-models, including model.onnx, model_quantized.onnx and more.
- 12 ONNX models (.onnx files)
- 104 files, including 71 .json, 12 .onnx, 9 .txt, 4 .py, 2 .md, 2 .ipynb
- Last updated 2025-04-28
ONNX files
| File | Size | Download |
|---|---|---|
| onnx_models/bloom_onnx/model.onnx | — | Download |
| onnx_models/bloom_onnx_quantized/model_quantized.onnx | — | Download |
| onnx_models/falcon_onnx/model.onnx | — | Download |
| onnx_models/gpt2_onnx/model.onnx | — | Download |
| onnx_models/gpt2_onnx_quantized/model_quantized.onnx | — | Download |
| onnx_models/opt_onnx/model.onnx | — | Download |
| onnx_models/opt_onnx_quantized/model_quantized.onnx | — | Download |
| onnx_models/phi_onnx/model.onnx | — | Download |
| onnx_models/qwen_onnx/model.onnx | — | Download |
| onnx_models/qwen_onnx_quantized/model_quantized.onnx | — | Download |
Showing 10 of 12 .onnx files — see the full file list on the right.
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
🚀 LLM to ONNX Converter
Convert small language models to ONNX format with guaranteed reliability for RAG and chatbot applications on resource-constrained hardware.
📋 Overview
This repository provides scripts to convert small language models to ONNX format and create INT8 quantized versions for efficient deployment on resource-constrained devices. Perfect for mobile applications, Unity game engines, and embedded systems.
✅ Tested Models
We've successfully tested the following models with example outputs:
| Model | Size | Quantized | Response Quality | Speed (sec) |
|---|---|---|---|---|
| Qwen-0.5B | 500M | ✅ | ❌ Poor | 8.37 |
| Qwen-0.5B | 500M | ❌ | ✅ Good | 15.69 |
| TinyLlama-1.1B | 1.1B | ✅ | ❌ Poor | 10.15 |
| TinyLlama-1.1B | 1.1B | ❌ | ✅ Good | 19.23 |
| Phi-1.5 | 1.3B | ❌ | ✅ Good | 15.32 |
| Falcon-RW-1B | 1B | ❌ | ✅ Good | 21.56 |
| GPT2-Medium | 355M | ✅ | ✅ Good | 6.27 |
| GPT2-Medium | 355M | ❌ | ✅ Good | 12.77 |
| OPT-350M | 350M | ✅ | ✅ Good | 4.33 |
| OPT-350M | 350M | ❌ | ✅ Good | 10.42 |
| Bloom-560M | 560M | ✅ | ❌ Poor | 11.93 |
| Bloom-560M | 560M | ❌ | ✅ Good | 34.38 |
🌟 Recommendations
Based on our testing:
- For best speed + quality: OPT-350M (quantized) - fastest with good quality
- For best overall quality: Phi-1.5 (non-quantized) - excellent responses
- For smallest size: GPT2-Medium or OPT-350M (quantized) - small with good performance
🚩 Key Findings
- Quantization provides ~2x speed improvement
- Smaller models (350-500M) quantize better than larger models (1B+)
- Some architectures (OPT, GPT2) handle quantization better than others
📁 Repository Structure
onnx_models/
├── bloom_onnx/
├── bloom_onnx_quantized/
├── falcon_onnx/
├── gpt2_onnx/
├── gpt2_onnx_quantized/
├── opt_onnx/
├── opt_onnx_quantized/
├── phi_onnx/
├── qwen_onnx/
├── qwen_onnx_quantized/
├── tinyllama_onnx/
└── tinyllama_onnx_quantized/
📚 Requirements
- Python 3.8+
- optimum
- onnxruntime
- transformers
- numpy
Model info
onnx
model
quantization
RAG
chatbot
NLP
small-models
mobile-deployment
embedded-systems
en
license:apache-2.0
region:us
↓ 0
♥ 0
Created: 2025-04-22
Updated: 2025-04-28
View on Hugging FaceFiles (104)
.gitattributes
.gitignore
CMD.md
README.md
hf_upload.ipynb
old_scripts/convert_for_unity.py
old_scripts/convert_single_model.py
old_scripts/convert_to_onnx.py
old_scripts/test_chat.py
onnx_models/bloom_onnx/config.json
onnx_models/bloom_onnx/generation_config.json
onnx_models/bloom_onnx/model.onnx
ONNX
onnx_models/bloom_onnx/special_tokens_map.json
onnx_models/bloom_onnx/tokenizer.json
onnx_models/bloom_onnx/tokenizer_config.json
onnx_models/bloom_onnx_quantized/config.json
onnx_models/bloom_onnx_quantized/model_quantized.onnx
ONNX
onnx_models/bloom_onnx_quantized/ort_config.json
onnx_models/bloom_onnx_quantized/special_tokens_map.json
onnx_models/bloom_onnx_quantized/tokenizer.json
onnx_models/bloom_onnx_quantized/tokenizer_config.json
onnx_models/falcon_onnx/config.json
onnx_models/falcon_onnx/generation_config.json
onnx_models/falcon_onnx/merges.txt
onnx_models/falcon_onnx/model.onnx
ONNX
onnx_models/falcon_onnx/special_tokens_map.json
onnx_models/falcon_onnx/tokenizer.json
onnx_models/falcon_onnx/tokenizer_config.json
onnx_models/falcon_onnx/vocab.json
onnx_models/gpt2_onnx/config.json
onnx_models/gpt2_onnx/generation_config.json
onnx_models/gpt2_onnx/merges.txt
onnx_models/gpt2_onnx/model.onnx
ONNX
onnx_models/gpt2_onnx/special_tokens_map.json
onnx_models/gpt2_onnx/tokenizer.json
onnx_models/gpt2_onnx/tokenizer_config.json
onnx_models/gpt2_onnx/vocab.json
onnx_models/gpt2_onnx_quantized/config.json
onnx_models/gpt2_onnx_quantized/merges.txt
onnx_models/gpt2_onnx_quantized/model_quantized.onnx
ONNX
onnx_models/gpt2_onnx_quantized/ort_config.json
onnx_models/gpt2_onnx_quantized/special_tokens_map.json
onnx_models/gpt2_onnx_quantized/tokenizer.json
onnx_models/gpt2_onnx_quantized/tokenizer_config.json
onnx_models/gpt2_onnx_quantized/vocab.json
onnx_models/opt_onnx/config.json
onnx_models/opt_onnx/generation_config.json
onnx_models/opt_onnx/merges.txt
onnx_models/opt_onnx/model.onnx
ONNX
onnx_models/opt_onnx/special_tokens_map.json
onnx_models/opt_onnx/tokenizer.json
onnx_models/opt_onnx/tokenizer_config.json
onnx_models/opt_onnx/vocab.json
onnx_models/opt_onnx_quantized/config.json
onnx_models/opt_onnx_quantized/merges.txt
onnx_models/opt_onnx_quantized/model_quantized.onnx
ONNX
onnx_models/opt_onnx_quantized/ort_config.json
onnx_models/opt_onnx_quantized/special_tokens_map.json
onnx_models/opt_onnx_quantized/tokenizer.json
onnx_models/opt_onnx_quantized/tokenizer_config.json
onnx_models/opt_onnx_quantized/vocab.json
onnx_models/phi_onnx/added_tokens.json
onnx_models/phi_onnx/config.json
onnx_models/phi_onnx/generation_config.json
onnx_models/phi_onnx/merges.txt
onnx_models/phi_onnx/model.onnx
ONNX
onnx_models/phi_onnx/special_tokens_map.json
onnx_models/phi_onnx/tokenizer.json
onnx_models/phi_onnx/tokenizer_config.json
onnx_models/phi_onnx/vocab.json
onnx_models/qwen_onnx/added_tokens.json
onnx_models/qwen_onnx/config.json
onnx_models/qwen_onnx/generation_config.json
onnx_models/qwen_onnx/merges.txt
onnx_models/qwen_onnx/model.onnx
ONNX
onnx_models/qwen_onnx/special_tokens_map.json
onnx_models/qwen_onnx/tokenizer.json
onnx_models/qwen_onnx/tokenizer_config.json
onnx_models/qwen_onnx/vocab.json
onnx_models/qwen_onnx_quantized/added_tokens.json
onnx_models/qwen_onnx_quantized/config.json
onnx_models/qwen_onnx_quantized/merges.txt
onnx_models/qwen_onnx_quantized/model_quantized.onnx
ONNX
onnx_models/qwen_onnx_quantized/ort_config.json
onnx_models/qwen_onnx_quantized/special_tokens_map.json
onnx_models/qwen_onnx_quantized/tokenizer.json
onnx_models/qwen_onnx_quantized/tokenizer_config.json
onnx_models/qwen_onnx_quantized/vocab.json
onnx_models/tinyllama_onnx/config.json
onnx_models/tinyllama_onnx/generation_config.json
onnx_models/tinyllama_onnx/model.onnx
ONNX
onnx_models/tinyllama_onnx/special_tokens_map.json
onnx_models/tinyllama_onnx/tokenizer.json
onnx_models/tinyllama_onnx/tokenizer.model
onnx_models/tinyllama_onnx/tokenizer_config.json
onnx_models/tinyllama_onnx_quantized/config.json
onnx_models/tinyllama_onnx_quantized/model_quantized.onnx
ONNX
onnx_models/tinyllama_onnx_quantized/ort_config.json
onnx_models/tinyllama_onnx_quantized/special_tokens_map.json
onnx_models/tinyllama_onnx_quantized/tokenizer.json
onnx_models/tinyllama_onnx_quantized/tokenizer.model
onnx_models/tinyllama_onnx_quantized/tokenizer_config.json
requirements.txt
test_models.ipynb
Related models
More from agoor97
No other models by this author.
More models
No task type declared.