verbatim-hindi-asr-flex-sc

Built with Indic-Transcribe-Flex from Bodhan AI / AI4Bharat.

A quantized, re-exported derivative of bodhan-ai/indic-transcribe-flex (itself built on nvidia/canary-1b-v2, CC-BY-4.0) for offline CPU inference with ONNX Runtime. It is not an original model and is not affiliated with or endorsed by Bodhan AI, AI4Bharat, IIT Madras or NVIDIA. The repository name is deliberately neutral: it claims no right to the Bodhan AI, AI4Bharat, IIT Madras or Indic-Transcribe names or marks, which appear here only as the attribution the licence requires. See NOTICE.md for full attribution.

What changed relative to the community int8 port adidsh/indic-transcribe-flex-int8-onnx: the decoder there re-runs the whole token prefix every step. Here it has a key/value cache, so each new token costs one position instead of the whole prefix. The weights are stored once — cross_kv.onnx (48 small projections) plus decoder_kv.onnx (everything else) are together the same size as the port's decoder.onnx, so the download (1.57 GB of ONNX) and memory stay at the port's level. Same int8 weights, same encoder, same transcripts (within noise), about 4–5x faster end to end.

Files

File	Bytes	SHA-256
encoder.onnx	1,121,847,999	3cdc9d8f1308a9d37023989fe712b8bdf87b7ada29135c71ca2d5c4bce360a65
cross_kv.onnx	50,926,641	42a23b58281d68a9408ed1cbefc5af4cf0a8c878e7840b8845670cfb89982c39
decoder_kv.onnx	396,918,392	4d08d0b94f96fb6ed8487e31721c9e320315dceb37ea0329cd60a6c4539b1d9f
bodhan_vocab.json	86,885	571fa2fac0efa0cb7c4196cb17e899560cf0c91510f7f2d739243d3766d5fb44
LICENSE	25,546	b39f2a9cb2d5ace3a03f4425cbc6db38ebd9a53ab7a7648d67dc1e0998ad75c2
NOTICE.md	2,895	b34e40da3159170b1f327121fc3bfc0f1dfa3bd35592629b7a62ebb6764a682a

All files are also listed in SHA256SUMS (shasum -a 256 -c SHA256SUMS).

Tensors (batch is always 1; L = 24 decoder layers, 8 heads, head_dim 128, vocab 7152)

encoder.onnx (unchanged from the port): feats f32 [1,128,T] → hidden_states f32 [1,T_enc,1024], T_enc ≈ T/8. No length or mask input — never pad.

cross_kv.onnx — run once per audio piece (shared by every prompt decoded on that piece)

• in encoder_hidden_states f32 [1,T_enc,1024] (the encoder's hidden_states)
• out cross_key.{l}, cross_value.{l} f32 [1,8,T_enc,128] for l = 0..23 (48 outputs)

decoder_kv.onnx — the prompt call, then one call per generated token

• in input_ids i64 [1,N]: the 10-token prompt on the first call (N = 10), then the token just chosen (N = 1)
• in past_self_key.{l}, past_self_value.{l} f32 [1,8,S,128] for l = 0..23: empty (S = 0) on the first call, afterwards the previous call's present_self_*
• in cross_key.{l}, cross_value.{l} f32 [1,8,T_enc,128] from cross_kv.onnx, unchanged for the piece
• out logits f32 [1,7152] (last position only)
• out present_self_key.{l}, present_self_value.{l} f32 [1,8,S+N,128]

The positions of the N new tokens are S..S+N−1, taken from the cache length inside the graph — there is no position input — and the self-attention mask is causal over the S+N keys, built in-graph. Keys are stored already divided by head_dim^0.25 (as the model does); callers never scale or reorder anything. Positions come from a fixed 1024-row table, so prompt + generated tokens must stay ≤ 1024. The empty first-call cache is an ordinary zero-length tensor (ONNX Runtime accepts it; with the Rust ort 2.0.0-rc.12 crate create it with Tensor::<f32>::new(&Allocator::default(), [1, 8, 0, 128]), since Tensor::from_array rejects zero dimensions).

Decoding

1. Features: 16 kHz mono; clips under 1 s are zero-padded to 1 s, centred. Pre-emphasis 0.97, reflect-pad 256, STFT n_fft 512 / hop 160 / symmetric Hann(400) centred in 512, power spectrum, 128-bin Slaney mel (librosa, fmin 0, fmax 8000), ln(x + 2^-24), then per-bin (x − mean)/(unbiased std + 1e-5) over the clip. No dither.
2. Prompt [7, 4, 18, L, L, 5, ITN, 11, 13, 15]: L = 89 Hindi, 66 English (else the index of <|xx|> in bodhan_vocab.json); ITN = 9 native script, 8 mixed (English words in Latin script). Hindi native [7,4,18,89,89,5,9,11,13,15], Hindi mixed [7,4,18,89,89,5,8,11,13,15].
3. encoder → cross_kv once. First decoder_kv call: the prompt with empty past_self_*; the argmax of logits is the first token. Then loop: stop on 3 (EOS) or 2 (PAD); otherwise append the token and call decoder_kv with it as input_ids [1,1], the previous present_self_* as past_self_* and the same cross_*, and take the argmax again. Cap: 512 new tokens and prompt + output ≤ 1024.
4. Text: concatenate bodhan_vocab.json[id] for the generated ids, replace ▁ with a space, trim.
5. Audio longer than 30 s: cut at pauses into ≤ 30 s pieces as the upstream long_form.py does.

Use the CPU execution provider (CoreML was slower and changed the output with the earlier port). Disabling the CPU memory arena for the cross_kv and decoder_kv sessions (the cache tensors grow by one position every call) cut peak memory by about 45 MB at no cost in speed; the figures below were measured that way.

Measured (Apple M4, 4 threads, ONNX Runtime 1.30 CPU, greedy)

WER / CER, compared with the community port's no-cache int8 decoder run through the same pipeline:

Test	No-cache int8 (port)	This repo
Student talk, first 2 min vs hand-checked reference, native	23.5% / 19.0%	23.5% / 19.0%
Student talk, first 2 min, mixed (English words in Latin)	28.0% / 24.3%	28.0% / 24.3%
FLEURS hi, 40 clips, native	9.8% / 3.9%	9.6% / 3.9%

4.6-minute Hindi student talk, whole file:

Decoder	Wall time (encoder + decoding)	Decoding	Peak process RAM
No-cache int8 (port)	58.3 s	52.3 s (62.4 ms/token)	2.65 GB
This repo	14.2 s	8.6 s (10.3 ms/token)	2.70 GB

4.1x faster overall, 6.1x faster decoding, at the same download size and about 0.05 GB more peak memory than the port (RAM is the whole Python measuring process, about 0.2 GB of it the interpreter). Over the whole 4.6 minutes the transcript differs from the no-cache one by 1 word in 607: activations are quantized per call, and a call now sees one token instead of the whole prefix.

Only Hindi (native and mixed) was measured for this derivative. The other languages in the metadata are what the upstream model supports; they were not re-tested here.

Intended use

Offline, on-device transcription for school reading assessment: a teacher's desktop app transcribes a student reading aloud and compares it with the text. No audio leaves the machine. Any automated judgement about a student should stay with a teacher (see the prohibited uses below). Not for safety-critical use.

Licence

Licensed under the Indic Open Model License v1.0 — the unaltered text is in LICENSE and governs. In short (this summary is not the licence):

• Free to use, modify and redistribute, including commercially, under this exact licence (Section 5).
• Anyone who makes this model or a derivative available to others must show "Built with Indic-Transcribe-Flex from Bodhan AI / AI4Bharat." in the product's user interface, or in top-level documentation if there is none, at least as prominently as any other third-party model credit (Section 2), and must not remove existing notices.
• Hosting it as an API or hosted service for third parties needs Bodhan AI's prior written approval, except for eligible public-interest entities or under the open-release waiver (Sections 3–4). Products with more than 500 million monthly users or US$250 million annual revenue need a separate licence (Section 14).
• No right to use Bodhan AI, AI4Bharat or IIT Madras names or marks beyond the attribution notice (Section 8.4).
• The base model nvidia/canary-1b-v2 is CC-BY-4.0; its attribution is in NOTICE.md.

10. Prohibited Uses

Regardless of whether Your use is otherwise licensed, exempt, or free of charge under this License — including use by an Eligible Public-Interest Entity — You must not use the Software, any Derivative, or any Output to:

• generate child sexual abuse material or any content that sexualizes minors;
• develop, market, or facilitate the development of weapons, including conventional weapons or chemical, biological, radiological, or nuclear (CBRN) weapons;
• conduct mass surveillance or build social-scoring systems;
• generate or disseminate disinformation, including for the purpose of election manipulation;
• make discriminatory automated decisions that materially affect a person's legal rights or access to essential services, without meaningful human oversight;
• impersonate a real, identifiable person's voice, image, or likeness without that person's consent, or otherwise generate non-consensual synthetic media of a real, identifiable person;
• power robocalls, auto-dialers, voice-phishing ("vishing"), or other fraudulent or deceptive automated calling or messaging campaigns; or
• power an AI companion product marketed or designed to simulate or encourage an exclusive romantic or intimate relationship, or to foster emotional dependency — provided that companionship, wellness, eldercare, mental-health-support, and language-learning products are excluded from this clause unless they are designed to simulate an exclusive romantic or intimate relationship.

Licensor may update or expand this list from time to time to address new or emerging misuse patterns that are materially similar in kind to those set out above (as updated, the "Prohibited Uses Policy"). Licensor will give at least 30 days' public notice of any update — for example, by posting the revised list in the Software's Hugging Face repository or at http://bodhan.ai — before it takes effect. An update applies to Your use of the Software and any Derivative from its stated effective date onward; it does not apply retroactively to conduct that occurred, or a Derivative that was Made Available, before that date.

Citation (upstream)

@misc{indictranscribe2026,
  title  = {Indic-Transcribe: Built for the way India actually speaks},
  author = {Bodhan AI, AI4Bharat},
  year   = {2026},
  url    = {https://bodhan.ai/research/blogs/indic-transcribe}
}