llmware/tiny-llama-chat-onnx

By llmware · ↓ 3 · ♥ 1 · Updated 2024-10-31

What's in this repo

This page lists the files in llmware/tiny-llama-chat-onnx, including model.onnx.

1 ONNX models (.onnx files)
11 files, including 6 .json, 1 .md, 1 .onnx, 1 .data, 1 .model
Last updated 2024-10-31

ONNX files

File	Size	Download
model.onnx	—	Download

How to use

These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.

README

tiny-llama-chat-onnx

tiny-llama-chat-onnx is an ONNX int4 quantized version of TinyLlama-Chat, providing a very fast, very small inference implementation, optimized for AI PCs using Intel GPU, CPU and NPU.

tiny-llama-chat is the official chat finetuned version of tiny-llama.

Model Description

Developed by: TinyLlama
Quantized by: llmware
Model type: llama
Parameters: 1.1 billion
Model Parent: TinyLlama-1.1B-Chat-v1.0
Language(s) (NLP): English
License: Apache 2.0
Uses: Chat and general purpose LLM
RAG Benchmark Accuracy Score: NA
Quantization: int4

Model Card Contact

llmware on github

llmware on hf

llmware website

Model info

onnx llama green llmware-chat p1 base_model:TinyLlama/TinyLlama-1.1B-Chat-v1.0 base_model:quantized:TinyLlama/TinyLlama-1.1B-Chat-v1.0 license:apache-2.0 region:us

↓ 3 ♥ 1

Created: 2024-10-26

Updated: 2024-10-31

View on Hugging Face

Files (11)

.gitattributes

README.md

config.json

genai_config.json

hash_record_sha256.json

model.onnx ONNX

model.onnx.data

special_tokens_map.json

tokenizer.json

tokenizer.model

tokenizer_config.json

Related models

More models

No task type declared.