MediaTek-Research/BreezyVoice
By MediaTek-Research · ↓ 0 · ♥ 51 · Updated 2025-02-18
What's in this repo
This page lists the files in MediaTek-Research/BreezyVoice, including campplus.onnx, flow.decoder.estimator.fp32.onnx, speech_tokenizer_v1.onnx.
- 3 ONNX models (.onnx files)
- 154 files, including 78 .scm, 25 .dict, 5 .conf, 4 .def, 4 .pt, 3 .ngrambin
- Last updated 2025-02-18
ONNX files
| File | Size | Download |
|---|---|---|
| campplus.onnx | — | Download |
| flow.decoder.estimator.fp32.onnx | — | Download |
| speech_tokenizer_v1.onnx | — | Download |
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
BreezyVoice
🚀 Try out our interactive UI playground now! 🚀
Or visit one of these resources:
BreezyVoice: Adapting TTS for Taiwanese Mandarin with Enhanced Polyphone Disambiguation -- Challenges and Insights
BreezyVoice is a voice-cloning text-to-speech system specifically adapted for Taiwanese Mandarin, highlighting phonetic control abilities via auxiliary 注音 (bopomofo) inputs. BreezyVoice is partially derived from CosyVoice
BreezyVoice outperforms competing commercial services in terms of naturalness.
BreezyVoice excels at code-switching scenarios.
| Code-Switching Term Category | BreezyVoice | Z | Y | U | M |
|---|---|---|---|---|---|
| General Words | 8 | 5 | 8 | 8 | 7 |
| Entities | 9 | 6 | 4 | 7 | 4 |
| Abbreviations | 9 | 8 | 6 | 6 | 7 |
| Toponyms | 3 | 3 | 7 | 3 | 4 |
| Full Sentences | 7 | 7 | 8 | 5 | 3 |
How to Run
Running from GitHub following instructions automatically downloads the model for you
You can also run the model from a specified local path by cloning the model
git lfs install
git clone https://huggingface.co/MediaTek-Research/BreezyVoice
You can then use the model as outlined in the single_inference.py script on GitHub, specifying the local model path via the model_path parameter.
If you like our work, please cite:
@article{hsu2025breezyvoice,
title={BreezyVoice: Adapting TTS for Taiwanese Mandarin with Enhanced Polyphone Disambiguation--Challenges and Insights},
author={Hsu, Chan-Jan and Lin, Yi-Cheng and Lin, Chia-Chun and Chen, Wei-Chih and Chung, Ho Lam and Li, Chen-An and Chen, Yi-Chang and Yu, Chien-Yu and Lee, Ming-Ji and Chen, Chien-Cheng and others},
journal={arXiv preprint arXiv:2501.17790},
year={2025}
}
@article{hsu2025breeze,
title={The Breeze 2 Herd of Models: Traditional Chinese LLMs Based on Llama with Vision-Aware and Function-Calling Capabilities},
author={Hsu, Chan-Jan and Liu, Chia-Sheng and Chen, Meng-Hsi and Chen, Muxi and Hsu, Po-Chun and Chen, Yi-Chang and Shiu, Da-Shan},
journal={arXiv preprint arXiv:2501.13921},
year={2025}
}
@article{du2024cosyvoice,
title={Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens},
author={Du, Zhihao and Chen, Qian and Zhang, Shiliang and Hu, Kai and Lu, Heng and Yang, Yexin and Hu, Hangrui and Zheng, Siqi and Gu, Yue and Ma, Ziyang and others},
journal={arXiv preprint arXiv:2407.05407},
year={2024}
}
Model info
onnx
arxiv:2501.17790
arxiv:2501.13921
arxiv:2407.05407
license:apache-2.0
region:us
↓ 0
♥ 51
Created: 2025-01-21
Updated: 2025-02-18
View on Hugging FaceFiles (154)
.gitattributes
CosyVoice-ttsfrd/resource/festival/Sable.v0_2.dtd
CosyVoice-ttsfrd/resource/festival/Singing.v0_1.dtd
CosyVoice-ttsfrd/resource/festival/VCLocalRules
CosyVoice-ttsfrd/resource/festival/apml.scm
CosyVoice-ttsfrd/resource/festival/apml_f2bf0lr.scm
CosyVoice-ttsfrd/resource/festival/apml_kaldurtreeZ.scm
CosyVoice-ttsfrd/resource/festival/cart_aux.scm
CosyVoice-ttsfrd/resource/festival/clunits.scm
CosyVoice-ttsfrd/resource/festival/clunits_build.scm
CosyVoice-ttsfrd/resource/festival/cmusphinx2_phones.scm
CosyVoice-ttsfrd/resource/festival/cslush.scm
CosyVoice-ttsfrd/resource/festival/cstr.scm
CosyVoice-ttsfrd/resource/festival/darpa_phones.scm
CosyVoice-ttsfrd/resource/festival/dicts/cmu/COPYING
CosyVoice-ttsfrd/resource/festival/dicts/cmu/allowables.scm
CosyVoice-ttsfrd/resource/festival/dicts/cmu/cmu2ft
CosyVoice-ttsfrd/resource/festival/dicts/cmu/cmu_lts_rules.scm
CosyVoice-ttsfrd/resource/festival/dicts/cmu/cmudict-0.4.diff
CosyVoice-ttsfrd/resource/festival/dicts/cmu/cmudict-0.4.out
CosyVoice-ttsfrd/resource/festival/dicts/cmu/cmudict-0.4.scm
CosyVoice-ttsfrd/resource/festival/dicts/cmu/cmudict_compile.scm
CosyVoice-ttsfrd/resource/festival/dicts/cmu/cmudict_extensions.scm
CosyVoice-ttsfrd/resource/festival/dicts/cmu/cmulex.scm
CosyVoice-ttsfrd/resource/festival/dicts/oald/COPYING
CosyVoice-ttsfrd/resource/festival/dicts/oald/ali.oald.lex
CosyVoice-ttsfrd/resource/festival/dicts/oald/oald_extensions.scm
CosyVoice-ttsfrd/resource/festival/dicts/oald/oald_lts_rules.scm
CosyVoice-ttsfrd/resource/festival/dicts/oald/oaldlex.scm
CosyVoice-ttsfrd/resource/festival/dicts/wsj.wp39.poslexR
CosyVoice-ttsfrd/resource/festival/dicts/wsj.wp39.tri.ngrambin
CosyVoice-ttsfrd/resource/festival/display.scm
CosyVoice-ttsfrd/resource/festival/duration.scm
CosyVoice-ttsfrd/resource/festival/email-mode.scm
CosyVoice-ttsfrd/resource/festival/engmorph.scm
CosyVoice-ttsfrd/resource/festival/engmorphsyn.scm
CosyVoice-ttsfrd/resource/festival/etc/email_filter
CosyVoice-ttsfrd/resource/festival/f2bdurtreeZ.scm
CosyVoice-ttsfrd/resource/festival/f2bf0lr.scm
CosyVoice-ttsfrd/resource/festival/festdoc.scm
CosyVoice-ttsfrd/resource/festival/festival.el
CosyVoice-ttsfrd/resource/festival/festival.scm
CosyVoice-ttsfrd/resource/festival/festtest.scm
CosyVoice-ttsfrd/resource/festival/fringe.scm
CosyVoice-ttsfrd/resource/festival/gswdurtreeZ.scm
CosyVoice-ttsfrd/resource/festival/holmes_phones.scm
CosyVoice-ttsfrd/resource/festival/hts.scm
CosyVoice-ttsfrd/resource/festival/init.scm
CosyVoice-ttsfrd/resource/festival/intonation.scm
CosyVoice-ttsfrd/resource/festival/java.scm
CosyVoice-ttsfrd/resource/festival/klatt_durs.scm
CosyVoice-ttsfrd/resource/festival/languages.scm
CosyVoice-ttsfrd/resource/festival/lexicons.scm
CosyVoice-ttsfrd/resource/festival/lts.scm
CosyVoice-ttsfrd/resource/festival/lts_build.scm
CosyVoice-ttsfrd/resource/festival/mbrola.scm
CosyVoice-ttsfrd/resource/festival/mettree.scm
CosyVoice-ttsfrd/resource/festival/module_description.scm
CosyVoice-ttsfrd/resource/festival/mrpa_allophones.scm
CosyVoice-ttsfrd/resource/festival/mrpa_durs.scm
CosyVoice-ttsfrd/resource/festival/mrpa_phones.scm
CosyVoice-ttsfrd/resource/festival/multisyn/multisyn.scm
CosyVoice-ttsfrd/resource/festival/multisyn/multisyn_pauses.scm
CosyVoice-ttsfrd/resource/festival/multisyn/radio_phones_multisyn.scm
CosyVoice-ttsfrd/resource/festival/multisyn/send_xwaves.scm
CosyVoice-ttsfrd/resource/festival/multisyn/target_cost.scm
CosyVoice-ttsfrd/resource/festival/ogimarkup-mode.scm
CosyVoice-ttsfrd/resource/festival/pauses.scm
CosyVoice-ttsfrd/resource/festival/phoneset.scm
CosyVoice-ttsfrd/resource/festival/phrase.scm
CosyVoice-ttsfrd/resource/festival/pos.scm
CosyVoice-ttsfrd/resource/festival/postlex.scm
CosyVoice-ttsfrd/resource/festival/radio_phones.scm
CosyVoice-ttsfrd/resource/festival/sable-latin.ent
CosyVoice-ttsfrd/resource/festival/sable-mode.scm
CosyVoice-ttsfrd/resource/festival/scfg.scm
CosyVoice-ttsfrd/resource/festival/scfg_wsj_wp20.gram
CosyVoice-ttsfrd/resource/festival/sec.B.hept.ngrambin
CosyVoice-ttsfrd/resource/festival/sec.ts20.quad.ngrambin
CosyVoice-ttsfrd/resource/festival/singing-mode.scm
CosyVoice-ttsfrd/resource/festival/siod.scm
CosyVoice-ttsfrd/resource/festival/siteinit.scm
CosyVoice-ttsfrd/resource/festival/soleml-mode.scm
CosyVoice-ttsfrd/resource/festival/speech.properties
CosyVoice-ttsfrd/resource/festival/synthesis.scm
CosyVoice-ttsfrd/resource/festival/tilt.scm
CosyVoice-ttsfrd/resource/festival/tobi.scm
CosyVoice-ttsfrd/resource/festival/tobi_rules.scm
CosyVoice-ttsfrd/resource/festival/token.scm
CosyVoice-ttsfrd/resource/festival/tokenpos.scm
CosyVoice-ttsfrd/resource/festival/tts.scm
CosyVoice-ttsfrd/resource/festival/unilex_phones.scm
CosyVoice-ttsfrd/resource/festival/voices.scm
CosyVoice-ttsfrd/resource/festival/voices/english/kal_diphone/COPYING
CosyVoice-ttsfrd/resource/festival/voices/english/kal_diphone/festvox/kal_diphone.scm
CosyVoice-ttsfrd/resource/festival/voices/english/kal_diphone/festvox/kaldurtreeZ.scm
CosyVoice-ttsfrd/resource/festival/voices/english/kal_diphone/group/kallpc16k.group
CosyVoice-ttsfrd/resource/festival/voices/english/rab_diphone/COPYING
CosyVoice-ttsfrd/resource/festival/voices/english/rab_diphone/festvox/rab_diphone.scm
CosyVoice-ttsfrd/resource/festival/voices/english/rab_diphone/group/rablpc16k.group
CosyVoice-ttsfrd/resource/festival/web.scm
CosyVoice-ttsfrd/resource/jprsc/COPYING
CosyVoice-ttsfrd/resource/jprsc/char.bin
CosyVoice-ttsfrd/resource/jprsc/left-id.def
CosyVoice-ttsfrd/resource/jprsc/matrix.bin
CosyVoice-ttsfrd/resource/jprsc/pos-id.def
CosyVoice-ttsfrd/resource/jprsc/rewrite.def
CosyVoice-ttsfrd/resource/jprsc/right-id.def
CosyVoice-ttsfrd/resource/jprsc/sys.dic
CosyVoice-ttsfrd/resource/jprsc/unk.dic
CosyVoice-ttsfrd/resource/languagedata_embedded.bin
CosyVoice-ttsfrd/resource/ws/AliTokenizer_example.conf
CosyVoice-ttsfrd/resource/ws/crf_model
CosyVoice-ttsfrd/resource/ws/ivr.dict
CosyVoice-ttsfrd/resource/ws/pos_hmm.dict
CosyVoice-ttsfrd/resource/ws/puncNumberTimeLetter.dict
CosyVoice-ttsfrd/resource/ws/ws-dfa-rule.dict
CosyVoice-ttsfrd/resource/ws/xiaoyan.dict
CosyVoice-ttsfrd/resource/ws_chhk/AliTokenizer_example.conf
CosyVoice-ttsfrd/resource/ws_chhk/ivr.dict
CosyVoice-ttsfrd/resource/ws_chhk/pos_hmm.dict
CosyVoice-ttsfrd/resource/ws_chhk/puncNumberTimeLetter.dict
CosyVoice-ttsfrd/resource/ws_chhk/ws-dfa-rule.dict
CosyVoice-ttsfrd/resource/ws_chhk/xiaoyan.dict
CosyVoice-ttsfrd/resource/ws_chsh/AliTokenizer_example.conf
CosyVoice-ttsfrd/resource/ws_chsh/ivr.dict
CosyVoice-ttsfrd/resource/ws_chsh/pos_hmm.dict
CosyVoice-ttsfrd/resource/ws_chsh/puncNumberTimeLetter.dict
CosyVoice-ttsfrd/resource/ws_chsh/ws-dfa-rule.dict
CosyVoice-ttsfrd/resource/ws_chsh/xiaoyan.dict
CosyVoice-ttsfrd/resource/ws_hk/AliTokenizer_example.conf
CosyVoice-ttsfrd/resource/ws_hk/ivr.dict
CosyVoice-ttsfrd/resource/ws_hk/pos_hmm.dict
CosyVoice-ttsfrd/resource/ws_hk/puncNumberTimeLetter.dict
CosyVoice-ttsfrd/resource/ws_hk/ws-dfa-rule.dict
CosyVoice-ttsfrd/resource/ws_hk/xiaoyan.dict
CosyVoice-ttsfrd/resource/ws_zhsc/AliTokenizer_example.conf
CosyVoice-ttsfrd/resource/ws_zhsc/crf_model
CosyVoice-ttsfrd/resource/ws_zhsc/ivr.dict
CosyVoice-ttsfrd/resource/ws_zhsc/pos_hmm.dict
CosyVoice-ttsfrd/resource/ws_zhsc/puncNumberTimeLetter.dict
CosyVoice-ttsfrd/resource/ws_zhsc/ws-dfa-rule.dict
CosyVoice-ttsfrd/resource/ws_zhsc/xiaoyan.dict
README.md
campplus.onnx
ONNX
configuration.json
cosyvoice.yaml
flow.decoder.estimator.fp32.onnx
ONNX
flow.encoder.fp32.zip
flow.pt
hift.pt
llm.pt
speech_tokenizer_v1.onnx
ONNX
spk2info.pt
Related models
More from MediaTek-Research
More models
No task type declared.