nkarv/whisper-large-v3-turbo-onnx-int8
By nkarv · ↓ 4 · ♥ 0 · Updated 2026-02-11
What's in this repo
This page lists the files in nkarv/whisper-large-v3-turbo-onnx-int8, including decoder_model_merged.onnx, encoder_model.onnx.
- 2 ONNX models (.onnx files)
- 17 files, including 10 .json, 2 .onnx, 2 .data, 1 .md, 1 .txt
- Last updated 2026-02-11
ONNX files
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
Whisper Large v3 Turbo - ONNX Dynamic INT8
Dynamically quantized ONNX version of openai/whisper-large-v3-turbo.
Quantization Details
| Property | Value |
|---|---|
| Method | Dynamic INT8 via ONNX Runtime |
| Target ops | MatMul-family graph ops selected by ONNX Runtime quantizer |
| Excluded ops | Conv |
| Calibration | None (dynamic quantization) |
| Tools | HuggingFace Optimum + ONNX Runtime |
| Compression ratio | 2.165x |
Usage
from optimum.onnxruntime import ORTModelForSpeechSeq2Seq
from transformers import AutoProcessor, pipeline
model = ORTModelForSpeechSeq2Seq.from_pretrained(
"nkarv/whisper-large-v3-turbo-onnx-int8"
)
processor = AutoProcessor.from_pretrained(
"nkarv/whisper-large-v3-turbo-onnx-int8"
)
pipe = pipeline(
"automatic-speech-recognition",
model=model,
tokenizer=processor.tokenizer,
feature_extractor=processor.feature_extractor,
)
result = pipe("audio.mp3", return_timestamps=True)
print(result["text"])
Why Dynamic INT8?
Dynamic quantization converts Linear layer weights to INT8 offline while computing activation scales at inference time. This avoids the need for a calibration dataset while still achieving significant compression and practical latency gains.
Model info
Created: 2026-02-11
Updated: 2026-02-11
View on Hugging FaceFiles (17)
Related models
More from nkarv
No other models by this author.