hari31416/indic-transcribe-onnx-int8

Indic-Transcribe ONNX INT8 (Selective MatMul)

This repository provides an optimized dynamic INT8 ONNX Runtime export of bodhan-ai/indic-transcribe-core. Using selective MatMul-only quantization and signed per-channel scaling, it preserves full Conformer convolution fidelity while cutting disk footprint by 67.6% and memory by 50.6%.

bodhan-ai/indic-transcribe-core is a 1-billion parameter multilingual ASR model based on NVIDIA's Canary-2 FastConformer AED (Audio Encoder-Decoder) architecture, natively supporting 25+ Indic languages and English.

Precision Variants

Repository	Precision	Storage Footprint	Peak Inference RAM	Accuracy / Parity
hari31416/indic-transcribe-onnx-fp32	FP32	4.59 GB (Baseline reference)	5,154 MB (~5.15 GB)	Bit-identical reference fidelity
hari31416/indic-transcribe-onnx-int8 (This repository)	INT8 (Selective MatMul)	1.49 GB (67.6% reduction)	2,528 MB (~2.53 GB)	Lossless transcription (100% exact match)

Storage and Memory Requirements

Metric	PyTorch Baseline	ONNX FP32	ONNX INT8 (Selective)	INT8 Reduction
Disk Footprint	4.64 GB (4,868 MB)	4.59 GB (4,701 MB)	1.49 GB (1,524 MB)	67.6% savings
Loaded RAM	358 MB (mmap)	4,585 MB	2,434 MB	46.9% savings
Peak Inference RAM	5,123 MB (~5.12 GB)	5,154 MB (~5.15 GB)	2,528 MB (~2.53 GB)	50.6% savings (~2.6 GB saved)

Measured via resident set size (RSS) tracking in isolated processes during warm end-to-end transcription on authentic 16 kHz Indic speech.

Empirical Parity and Accuracy Benchmarks

Evaluated against the PyTorch reference baseline on authentic native Indic speech clips from Google FLEURS.

Overall Performance

• Number of languages evaluated: 5 (Hindi, Tamil, Bengali, Marathi, Telugu)
• Total audio duration: 59.22s
• Average Encoder Cosine Similarity: 0.996358
• Exact Text Parity Rate: 5/5 (100% exact match)
• Mean WER vs PyTorch Baseline: 0.00%
• Mean CER vs PyTorch Baseline: 0.00%
• Cumulative Latency: 6.11s (vs PyTorch 13.87s)
• Average Real-Time Factor (RTF): 9.69x (2.27x faster than PyTorch)

Multi-Language Evaluation

Language	Audio Duration	Enc Cos Sim	WER vs PT	CER vs PT	Exact Match	Latency	RTFx	Speedup vs PT
Hindi (hi)	12.2s	0.995284	0.0%	0.0%	PASS	1.19s	10.32x	4.41x
Tamil (ta)	14.1s	0.993200	0.0%	0.0%	PASS	1.81s	7.81x	1.45x
Bengali (bn)	6.6s	0.999441	0.0%	0.0%	PASS	0.53s	12.38x	3.18x
Marathi (mr)	15.3s	0.998828	0.0%	0.0%	PASS	1.40s	10.92x	1.54x
Telugu (te)	11.0s	0.995039	0.0%	0.0%	PASS	1.18s	9.27x	1.84x

Architecture Overview

• Audio Front-End:
  • Sample rate: 16,000 Hz mono.
  • Mel spectrogram: 128 filterbanks with 25 ms window, 10 ms hop size, 512-point FFT, per-feature normalization.
• FastConformer Encoder:
  • 32 Conformer blocks with depthwise striding (8x subsampling).
  • Relative positional encoding, 8 attention heads, $d_{\text{model}} = 1024$, $FFN = 4096$.
  • Output artifact: encoder-model.onnx.
• Transformer Decoder:
  • 24 pre-LN Transformer decoder layers with cross-attention.
  • 8 attention heads, $d_{\text{model}} = 1024$, $FFN = 4096$.
  • Vocabulary: 7,152 tokens (1,152 special/timestamp tokens + 6,000 multilingual SPM tokens).
  • Stateful decoder_mems cache for autoregressive sequence generation.
  • Output artifact: decoder-model.onnx.

Installation

Create a virtual environment and install the lightweight runtime requirements:

pip install onnxruntime soundfile numpy torch

Quick Start Examples

1. Using transcribe.py CLI

# Transcribe Hindi audio in native script
python transcribe.py --audio audio.wav --lang hi --mode native

# Transcribe Tamil with numbers formatted in English digits (mixed ITN)
python transcribe.py --audio audio.wav --lang ta --mode mixed

# Transcribe Bengali in Latin transliteration
python transcribe.py --audio audio.wav --lang bn --mode romanised

2. Using Pure ONNX Runtime Pipeline in Python

from pathlib import Path
from pipeline_onnx import IndicTranscribeONNX

# Initialize ONNX pipeline
pipeline = IndicTranscribeONNX(Path("./"))

# Transcribe audio
transcript = pipeline.transcribe("audio.wav", lang="hi", mode="native")
print("Transcription:", transcript)

3. Using with onnx-asr

import onnx_asr

# Option A: Directly from Hugging Face Hub
model = onnx_asr.load_model("hari31416/indic-transcribe-onnx-int8")

# Option B: From local directory
model = onnx_asr.load_model("nemo-canary-1b-v2", path="./")

transcript = model.recognize("audio.wav")
print("Transcription:", transcript)

Model Files in This Repository

• encoder-model.onnx: FastConformer audio encoder.
• decoder-model.onnx: Transformer autoregressive text decoder.
• vocab.txt: 7,152 token vocabulary mapping.
• config.json: Canary architecture configuration.
• feature_extractor.safetensors: Mel spectrogram filterbanks and window weights.
• feature_extraction_indic_canary.py: Mel front-end feature extractor matching NeMo.
• pipeline_onnx.py: Standalone ONNX Runtime inference engine.
• transcribe.py: Simple command-line transcription utility.
• requirements.txt: Python runtime dependencies.
• .gitattributes: Git LFS tracking configuration for large ONNX models.

Citation

@misc{indic_transcribe_core,
  title={Indic-Transcribe-Core: Multilingual Indic Speech Recognition in ONNX},
  author={Bodhan AI},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/bodhan-ai/indic-transcribe-core}}
}