ebrinz/chatterbox-turbo-coreml
By ebrinz · ↓ 0 · ♥ 0 · Updated 2026-03-21
What's in this repo
This page lists the files in ebrinz/chatterbox-turbo-coreml, including conditional_decoder_single.onnx, language_model_single.onnx.
- 2 ONNX models (.onnx files)
- 41 files, including 15 .bin, 10 .json, 5 .mil, 4 .npy, 2 .safetensors, 2 .onnx
- Last updated 2026-03-21
ONNX files
| File | Size | Download |
|---|---|---|
| onnx/conditional_decoder_single.onnx | — | Download |
| onnx/language_model_single.onnx | — | Download |
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
Chatterbox Turbo CoreML+ONNX
On-device text-to-speech for Apple Silicon. Hybrid CoreML + ONNX Runtime pipeline — ~1.5s latency, 11x faster than MLX.
Based on ResembleAI/chatterbox-turbo.
Benchmarks (Apple M3 Pro)
| Stage | Time | Engine |
|---|---|---|
| T3 Prefill (380 tokens) | ~170ms | CoreML (CPU+GPU) |
| T3 Decode (per token) | ~43ms | ONNX Runtime C API |
| Conditional Decoder | ~850ms | ONNX Runtime |
| Total (~20 tokens) | ~1.5s | |
| Short phrase | ~1.0s |
Comparison
| Runtime | Total latency | vs MLX |
|---|---|---|
| This model | ~1.5s | 11x faster |
| ResembleAI ONNX (Python) | ~1.3s | 13x faster |
| CoreML only (our earlier approach) | ~5.5s | 3x faster |
| MLX 8-bit (macOS) | ~17s | baseline |
Optimization Journey
| Version | Latency | Key Change |
|---|---|---|
| v1: CoreML only | 5.5s | First working pipeline |
| v2: + ONNX decode | 2.5s | Hybrid CoreML prefill + ONNX decode |
| v3: + C API | 2.5s | Zero-copy tensors (43ms/tok) |
| v4: + conditional_decoder | ~1.5s | Single ONNX call replaces S3+vocoder |
Architecture
Two-stage pipeline:
Text → [GPT-2 BPE Tokenizer]
→ [T3 Prefill: CoreML] (speaker + text + speech conditioning)
→ [T3 Decode: ONNX C API] (autoregressive speech token generation)
→ [Conditional Decoder: ONNX] (speech tokens → waveform)
→ 24kHz Audio
- T3 Prefill (CoreML): Handles the full Chatterbox conditioning — speaker projection, text embedding (50k vocab), speech embedding (6.5k vocab), and conditioning prompt tokens
- T3 Decode (ONNX Runtime C API): Zero-copy KV cache for fast autoregressive generation at 43ms/token
- Conditional Decoder (ONNX Runtime): Replaces 5 separate stages (S3 encoder, U-Net denoiser, flow matching, vocoder, ISTFT) with a single model call
Model Files
| File | Size | Description |
|---|---|---|
T3Prefill.mlmodelc/ |
1.4 GB | CoreML GPT-2 prefill with conditioning |
onnx/language_model_single.onnx |
1.2 GB | ONNX GPT-2 decode with KV cache |
onnx/conditional_decoder_single.onnx |
770 MB | ONNX S3+vocoder (tokens → waveform) |
speech_emb.npy |
26 MB | Speech token embedding table |
default-conds.safetensors |
161 KB | Default voice profile |
tokenizer.json |
3.4 MB | GPT-2 BPE tokenizer |
Quick Start (Swift)
import ChatterboxCoreML
let model = try await ChatterboxCoreMLModel.load(from: modelDirectory)
let audio = try await model.generate("Hello there.")
// audio is AVAudioPCMBuffer at 24kHz
Requirements
- macOS 15+ / iOS 18+
- Apple Silicon (M1/M2/M3/M4)
- Swift 6.2+
onnxruntime-swift-package-manager(1.24+)swift-transformers,swift-safetensors
Voice Cloning
let voice = URL(filePath: "path/to/custom-conds.safetensors")
let audio = try await model.generate("Hello there.", voice: voice)
License
MIT (same as Chatterbox Turbo)
Model info
onnx
text-to-speech
speech
voice-cloning
coreml
apple-silicon
en
base_model:ResembleAI/chatterbox-turbo
base_model:quantized:ResembleAI/chatterbox-turbo
license:mit
region:us
↓ 0
♥ 0
Created: 2026-03-17
Updated: 2026-03-21
View on Hugging FaceFiles (41)
.gitattributes
FullVocoder.mlmodelc/analytics/coremldata.bin
FullVocoder.mlmodelc/coremldata.bin
FullVocoder.mlmodelc/metadata.json
FullVocoder.mlmodelc/model.mil
FullVocoder.mlmodelc/weights/weight.bin
README.md
S3Encoder.mlmodelc/analytics/coremldata.bin
S3Encoder.mlmodelc/coremldata.bin
S3Encoder.mlmodelc/metadata.json
S3Encoder.mlmodelc/model.mil
S3Encoder.mlmodelc/weights/weight.bin
S3UNet.mlmodelc/analytics/coremldata.bin
S3UNet.mlmodelc/coremldata.bin
S3UNet.mlmodelc/metadata.json
S3UNet.mlmodelc/model.mil
S3UNet.mlmodelc/weights/weight.bin
T3Prefill.mlmodelc/analytics/coremldata.bin
T3Prefill.mlmodelc/coremldata.bin
T3Prefill.mlmodelc/metadata.json
T3Prefill.mlmodelc/model.mil
T3Prefill.mlmodelc/weights/weight.bin
T3Stateful.mlmodelc/analytics/coremldata.bin
T3Stateful.mlmodelc/coremldata.bin
T3Stateful.mlmodelc/metadata.json
T3Stateful.mlmodelc/model.mil
T3Stateful.mlmodelc/weights/weight.bin
added_tokens.json
default-conds.safetensors
hift_vocoder.safetensors
merges.txt
onnx/conditional_decoder_single.onnx
ONNX
onnx/language_model_single.onnx
ONNX
special_tokens_map.json
speech_emb.npy
spkr_enc_bias.npy
spkr_enc_weight.npy
text_emb.npy
tokenizer.json
tokenizer_config.json
vocab.json
Related models
More from ebrinz
No other models by this author.