kangkyu/icefall-asr-korean-multidomain-zipformer-2026-03-19

Korean Multi-Domain Zipformer ASR

A Korean ASR model finetuned on 6,863 hours across 3 domains from AI Hub, built on KsponSpeech pretrained weights using the icefall Zipformer transducer (74.78M parameters).

Key Feature: Korean-Only Output

This model outputs Korean text exclusively. All English words, numbers, and abbreviations are transcribed as their Korean pronunciations — exactly as spoken.

The training data uses (written)/(spoken) pronunciation pairs where the spoken form (Korean) is the ground truth. This eliminates ambiguity — the model transcribes exactly what it hears.

ITN (Inverse Text Normalization) is recommended for display. We provide ITN dictionaries with 14,000+ written↔spoken pairs to convert Korean output back to mixed-script text (e.g. 육십 분 → 60분).

Performance

CER (Character Error Rate), spaces excluded. Our model measured with sherpa-onnx int8 greedy decoding on CPU.

Benchmark source: Awesome Korean Speech Recognition (2026-03)

Our model with LM rescoring surpasses Naver ClovaSpeech on KSP eval_clean (7.99% vs 8.05%) while being fully open-source.

With RNN-LM Rescoring (scale=0.2)

Training Data

Text was cleaned using domain-specific rules to handle transcription markers:

• (written)/(spoken) pronunciation pairs → spoken form retained
• n/ b/ o/ speaker/noise/overlap markers → removed
• + stuttering → pre-correction syllables removed
• Filler words (아/, 어/) → slash removed, Korean retained
• PII masking (**, OOO) → entire utterance excluded

Model Architecture

How to Use

With sherpa-onnx (Recommended)

pip install sherpa-onnx

git lfs install
git clone https://huggingface.co/kangkyu/icefall-asr-korean-multidomain-zipformer-2026-03-19
cd icefall-asr-korean-multidomain-zipformer-2026-03-19

import sherpa_onnx

recognizer = sherpa_onnx.OfflineRecognizer.from_transducer(
    encoder="encoder-epoch-1-avg-1.int8.onnx",
    decoder="decoder-epoch-1-avg-1.int8.onnx",
    joiner="joiner-epoch-1-avg-1.int8.onnx",
    tokens="tokens.txt",
    num_threads=4,
    decoding_method="greedy_search",
)

stream = recognizer.create_stream()
stream.accept_wave_file("test_wavs/0.wav")
recognizer.decode(stream)
print(stream.result.text)

With RNN-LM Rescoring

recognizer = sherpa_onnx.OfflineRecognizer.from_transducer(
    encoder="encoder-epoch-1-avg-1.int8.onnx",
    decoder="decoder-epoch-1-avg-1.int8.onnx",
    joiner="joiner-epoch-1-avg-1.int8.onnx",
    tokens="tokens.txt",
    num_threads=4,
    decoding_method="modified_beam_search",
    lm="data/lm/no-state-epoch-9-avg-3.onnx",
    lm_scale=0.2,
)

Files

.
├── encoder-epoch-1-avg-1.onnx          # Encoder FP32 (261MB)
├── encoder-epoch-1-avg-1.int8.onnx     # Encoder INT8 (71MB) ← recommended
├── decoder-epoch-1-avg-1.onnx          # Decoder FP32 (11MB)
├── decoder-epoch-1-avg-1.int8.onnx     # Decoder INT8 (3MB)
├── joiner-epoch-1-avg-1.onnx           # Joiner FP32 (10MB)
├── joiner-epoch-1-avg-1.int8.onnx      # Joiner INT8 (3MB)
├── tokens.txt                          # BPE token list
├── bpe.model                           # SentencePiece BPE model
├── exp/
│   └── pretrained.pt                   # PyTorch checkpoint (857MB)
├── data/
│   ├── lang_bpe_5000/
│   │   ├── bpe.model
│   │   └── tokens.txt
│   ├── lm/
│   │   ├── no-state-epoch-9-avg-3.onnx # RNN-LM for offline rescoring (37MB)
│   │   └── with-state-epoch-9-avg-3.onnx # RNN-LM for streaming (37MB)
│   └── itn_pairs/
│       ├── counseling.jsonl            # 3,943 pairs
│       └── meeting.jsonl               # 10,998 pairs
└── test_wavs/
    ├── 0.wav, 1.wav, 2.wav

RNN Language Model

6.7M-parameter RNN-LM for rescoring.

Use no-state-epoch-9-avg-3.onnx with OfflineRecognizer for rescoring.

Training Details

• Base model: johnBamma/icefall-asr-ksponspeech-zipformer-2024-06-24 (KsponSpeech 969h pretrained)
• Finetuning: 2 rounds on A40 GPU, pruned RNN-T loss, Eden LR scheduler
• Data format: Lhotse shar (sharded tar archives)
• Precision: BF16 mixed precision

Limitations

• Non-causal (offline) model — not suitable for real-time streaming. Use for batch/file transcription.
• Korean-only output — English words and numbers appear as Korean pronunciation. ITN post-processing needed for display.
• No punctuation — the model does not output punctuation marks.

License

Apache License 2.0.

This model was trained using AI datasets built as part of projects by the National Information Society Agency (NIA, 한국지능정보사회진흥원) of the Republic of Korea, provided through AI Hub.

Original datasets are not redistributed. Download directly from AI Hub to reproduce training.

Acknowledgments

• k2-fsa/icefall — Training framework
• k2-fsa/sherpa-onnx — Inference engine
• johnBamma — KsponSpeech pretrained baseline
• AI Hub (한국지능정보사회진흥원) — Training datasets