dewdev/bling-tiny-llama-onnx
By dewdev · ↓ 0 · ♥ 0 · Updated 2025-02-04
What's in this repo
This page lists the files in dewdev/bling-tiny-llama-onnx, including model.onnx, model_int8.onnx, model_q4.onnx and more.
- 4 ONNX models (.onnx files)
- 15 files, including 7 .json, 4 .onnx, 1 .md, 1 .data, 1 .model
- Last updated 2025-02-04
ONNX files
| File | Size | Download |
|---|---|---|
| model.onnx | — | Download |
| onnx/model_int8.onnx | — | Download |
| onnx/model_q4.onnx | — | Download |
| onnx/model_uint8.onnx | — | Download |
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
cloned from llmware/bling-tiny-llama-onnx
bling-tiny-llama-onnx
bling-tiny-llama-onnx is a very small, very fast fact-based question-answering model, designed for retrieval augmented generation (RAG) with complex business documents, quantized and packaged in ONNX int4 for AI PCs using Intel GPU, CPU and NPU.
This model is one of the smallest and fastest in the series. For higher accuracy, look at larger models in the BLING/DRAGON series.
Model Description
- Developed by: llmware
- Model type: tinyllama
- Parameters: 1.1 billion
- Quantization: int4
- Model Parent: llmware/bling-tiny-llama-v0
- Language(s) (NLP): English
- License: Apache 2.0
- Uses: Fact-based question-answering, RAG
- RAG Benchmark Accuracy Score: 86.5
Model Card Contact
Model info
Created: 2025-02-04
Updated: 2025-02-04
View on Hugging FaceFiles (15)
Related models
More from dewdev
More models
No task type declared.