llmware/llama-3.2-3b-onnx-qnn
By llmware · ↓ 10 · ♥ 1 · Updated 2025-02-14
What's in this repo
This page lists the files in llmware/llama-3.2-3b-onnx-qnn, including dequantizer.onnx, position-processor.onnx, prompt_1_of_3_qnn_ctx.onnx and more.
- 9 ONNX models (.onnx files)
- 20 files, including 9 .onnx, 6 .json, 3 .bin, 1 .md
- Last updated 2025-02-14
ONNX files
| File | Size | Download |
|---|---|---|
| dequantizer.onnx | — | Download |
| position-processor.onnx | — | Download |
| prompt_1_of_3_qnn_ctx.onnx | — | Download |
| prompt_2_of_3_qnn_ctx.onnx | — | Download |
| prompt_3_of_3_qnn_ctx.onnx | — | Download |
| quantizer.onnx | — | Download |
| token_1_of_3_qnn_ctx.onnx | — | Download |
| token_2_of_3_qnn_ctx.onnx | — | Download |
| token_3_of_3_qnn_ctx.onnx | — | Download |
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
llama-3.2-3b-onnx-qnn
llama-3.2-3b-onnx-qnn is an ONNX QNN int4 quantized version of Llama 3.2 3B Instruct, providing a very small, very fast inference implementation, optimized for NPU deployment on Windows ARM64 AI PCs with Snapdragon Elite X NPU processors.
llama-3.2-3b-instruct is a new 3B chat foundation model from Meta.
Model Description
- Developed by: meta-llama
- Quantized by: llmware
- Model type: llama-3.2
- Parameters: 3 billion
- Model Parent: meta-llama/Meta-Llama-3.2-3B-Instruct
- Language(s) (NLP): English
- License: Llama 3.2 Community License
- Uses: General chat use cases
- RAG Benchmark Accuracy Score: NA
- Quantization: int4
Model Card Contact
Model info
Created: 2025-02-12
Updated: 2025-02-14
View on Hugging FaceFiles (20)
Related models
More from llmware
More models
No task type declared.