Mazino0/fire-red-streaming-vad-onnx

FireRedVAD Stream-VAD (ONNX)

ONNX export of the Stream-VAD model from FireRedTeam/FireRedVAD for real-time streaming voice activity detection.

Model Details

Files

• firered_vad.onnx — The DFSMN model (2.3 MB)
• cmvn.json — CMVN normalization parameters (80-dim means + inverse stddevs)
• model_meta.json — Architecture metadata for runtime initialization

Input/Output Specification

Inputs

Outputs

Streaming Usage

Initialize caches as zeros with cache_len=10:

caches = {f"cache_{i}": np.zeros((1, 128, 10), dtype=np.float32) for i in range(8)}

For each audio chunk:

1. Extract 80-dim fbank features (25ms window, 10ms shift, 16kHz)
2. Apply CMVN normalization: (feature - mean) * inv_stddev
3. Run inference with feat + current caches
4. Carry new_cache_* outputs to the next call
5. Speech probability > 0.5 indicates speech

import onnxruntime as ort
import numpy as np

sess = ort.InferenceSession("firered_vad.onnx")
caches = {f"cache_{i}": np.zeros((1, 128, 10), dtype=np.float32) for i in range(8)}

# For each chunk of fbank features:
feat = extract_fbank(audio_chunk)  # [1, num_frames, 80]
feat = apply_cmvn(feat, cmvn)      # normalize
inputs = {"feat": feat, **caches}
outputs = sess.run(None, inputs)

probs = outputs[0]                  # [1, num_frames, 1]
for i in range(8):
    caches[f"cache_{i}"] = outputs[i + 1]

Export Script

This model was exported using scripts/export_firered_vad.py from the second-brain project. The script:

1. Downloads the official PyTorch weights from FireRedTeam/FireRedVAD
2. Wraps the model with flattened cache I/O for ONNX compatibility
3. Exports with dynamic axes for variable-length streaming input
4. Converts Kaldi CMVN ark → JSON

License

Apache 2.0, following the original FireRedTeam/FireRedVAD license.