Kokoro-82M — LiteRT (free-text, bucketed)

⚠️ Labeled preview — FP32, CPU. Arbitrary free text → speech (not a single baked sentence). The neural graphs are LiteRT .tflite; two small steps run host-side (the hn-NSF source STFT and the final iSTFT overlap-add). GPU + quantization are the next steps.

A LiteRT (.tflite) conversion of hexgrad/Kokoro-82M (StyleTTS2 + ISTFTNet) for on-device free-text text-to-speech (arbitrary text in, 24 kHz speech out). Audio fidelity ≈ 0.9994 magnitude-spectrogram correlation to the PyTorch reference (verified across multiple held-out sentences, not just the export sample).

Kokoro has one data-dependent length (the duration→alignment expansion L = sum(pred_dur)), which litert_torch cannot keep dynamic (the LSTM sequence axis specializes). It is therefore split into three fixed-bucket bundles with host steps between them, so arbitrary text works by left-padding to the bucket and trimming the output (longer text is split into sentences host-side, each ≤ the bucket):

text --(G2P, host)--> phoneme ids
1. kokoro_predictor.tflite : ids[1,128], ref_s[1,256], attn[1,128] -> duration, d, t_en
   host: pred_dur = round(duration); alignment one-hot aln[1,128,512]; frame_mask[1,512]
2. kokoro_prosody.tflite   : d, t_en, aln, ref_s, frame_mask       -> asr, F0, N
   host: har = STFT(SineGen(f0_upsamp(F0)))     (the hn-NSF excitation)
3. kokoro_vocoder.tflite   : asr, F0, N, har, ref_s, frame_mask    -> spec, phase
   host: iSTFT overlap-add(spec, phase) -> 24 kHz waveform; trim to L*600

Files

File	Precision	Size	Role
kokoro_predictor.tflite	fp32	~91 MB	PL-BERT + duration/text encoders (masked unrolled bi-LSTMs)
kokoro_prosody.tflite	fp32	~37 MB	shared prosody LSTM + F0/N (masked)
kokoro_vocoder.tflite	fp32	~236 MB	iSTFTNet decoder → magnitude/phase spectrogram
istft_Wr_f32.bin, istft_Wi_f32.bin	fp32	880 B each	inverse-DFT bases for the host-side iSTFT
kokoro_82m_fixedlen_fp32.tflite	fp32	~338 MB	single-graph fixed-length demo build used by the litert-samples text_to_speech sample (see below)

Token bucket T = 128, frame bucket L = 512 (≈ 12.8 s of audio per chunk at 24 kHz). Bundles are voice-independent — the voice is the ref_s input (a voices/*.pt from the base repo, indexed by token-sequence length).

Fixed-length demo build (litert-samples)

kokoro_82m_fixedlen_fp32.tflite is the earlier single-graph build (fixed-length input, baked demo sentence length, spec+phase out, host-side iSTFT) kept for the official litert-samples text_to_speech sample, which downloads it by this name. Measured on Pixel 8a (fp32, CPU, 4 threads): RTF ≈ 1.8 (~6.6 s to synthesize 3.7 s of audio) — quantization is the path to realtime on-device.

Specs

Task	Text-to-speech (English), free text, 24 kHz mono
Source	hexgrad/Kokoro-82M (StyleTTS2 + ISTFTNet)
Fidelity	magspec-corr 0.9994 vs PyTorch (waveform corr ≈ 0.98 — the bounded bucket pad-boundary effect; the spectrum is what's perceived)
Runtime	CPU (LiteRT CompiledModel API)

How it was converted

• Stock official converter (litert_torch), general path — Kokoro is StyleTTS2/ISTFTNet, not a transformer LLM, so the Generative-API re-authoring path does not apply.
• Three fixed-bucket bundles because the dynamic alignment length can't stay symbolic through the converter (the dynamic-LSTM wall). Every workaround is load-bearing and numerically faithful: the 6 bidirectional LSTMs are unrolled as masked bi-LSTMs that carry state through right-padding (a fused nn.LSTM leaks pad tokens into the backward pass and wrecks prosody); the 58 AdaIN InstanceNorms normalize over real frames only so bucket pad frames don't poison the statistics; the hn-NSF source STFT runs host-side (its atan2 phase flips at the F0→0 pad boundary on-device).
• iSTFT runs host-side (vocoder emits spec+phase): the in-graph conv-transpose iSTFT hits a converter weight-dedup bug that fuses the cos/sin DFT bases. The host overlap-add (no learned weights) is numerically exact.

Training data

Inherited from hexgrad/Kokoro-82M: a few hundred hours of permissive / non-copyrighted audio — public-domain audio, audio under permissive licenses (e.g. Koniwa tnc CC BY 3.0, SIWIS CC BY 4.0), and synthetic audio from large-provider TTS — paired with IPA phoneme labels. No proprietary or scraped personal recordings, no custom voice clones. This LiteRT artifact is a format conversion of the released weights and introduces no additional training data.

PII

No personally identifiable information is included. Per the base model's disclosure the training audio is permissive / public-domain / synthetic rather than scraped personal recordings; to the best of our knowledge the released weights contain no PII, and the conversion adds none.

Front-end (free-text G2P)

For arbitrary text with no dropped words (names, brands, numbers), pair with the neural grapheme-to-phoneme front-end: litert-community/Kokoro-G2P-en-US.

Sample / code

• Official Android sample (Interpreter API, fixed-length demo build): google-ai-edge/litert-samples #159
• Conversion / pipeline code: https://github.com/john-rocky/LiteRT-Models

Roadmap

• GPU: the attention's fused-QKV >4-D layout + mask EQUAL/SELECT keep it on CPU; decomposing attention to ≤4-D would let it ride the GPU delegate. In-graph iSTFT on GPU additionally needs an FFT kernel in the LiteRT GPU delegate (ML Drift).
• Quantization (int8/int4) is the obvious next step.

Status

Labeled preview — the converted model is parity-verified. The official litert-samples text_to_speech sample (fixed-length demo build) is in review at google-ai-edge/litert-samples #159; a free-text sample (3-bundle + G2P front-end) is in progress.

Performance

Measured on an Apple M4 Max, CPU/XNNPACK at 8 threads, ai-edge-litert 2.1.6 — median of 15 warm runs per graph, with zero-filled inputs of each graph's declared static shape. Run-to-run spread stayed within 5%.

Graph	Warm median	First call
kokoro_predictor.tflite	51.9 ms	417.1 ms
kokoro_prosody.tflite	38.9 ms	491.4 ms
kokoro_vocoder.tflite	396.1 ms	561.2 ms
kokoro_82m_fixedlen_fp32.tflite (demo build)	165.0 ms	415.8 ms

The three-bundle chain costs 487 ms of graph time per chunk, and the vocoder is 81% of it. That figure deliberately excludes the host-side iSTFT overlap-add that turns the spectrogram and phase into a waveform, and the free-text G2P front-end — both run outside these graphs — so it is a floor for the chunk, not an end-to-end synthesis time. No end-to-end RTF is quoted for this Mac because the host stages were not measured here.

The Pixel 8a figure quoted above for the fixed-length demo build (RTF ≈ 1.8, fp32, CPU, 4 threads) is an end-to-end number on a different device and includes those host stages, so it is not comparable to the per-graph column.

Android (Pixel 8a)

Android figures use the standard TFLite benchmark_model on a Pixel 8a (Tensor G3, Android 16) — 5 warm-up runs then 20 timed runs, CPU at 4 threads.

Graph	GPU (OpenCL)	CPU (XNNPACK, 4 threads)
kokoro_82m_fixedlen_fp32.tflite	did not run	5501 ms
kokoro_predictor.tflite	did not run	2432 ms
kokoro_prosody.tflite	did not run	539 ms
kokoro_vocoder.tflite	did not run	17067 ms

4 of these graphs do not load on the OpenCL delegate at all, so on this Pixel 8a the CPU column is the only number for them. Nothing here is faster on the GPU; run the three-bundle pipeline on the CPU on Android — the one accelerator row recorded since is the fixed-length demo build on a Galaxy S26 Hexagon NPU (see Snapdragon NPU (Hexagon) below).

Snapdragon NPU (Hexagon)

• kokoro_82m_fixedlen_fp32.tflite — the NPU runs it at 1150.4 ms. The GPU does not — the benchmark process was killed, most likely out of memory.
• kokoro_predictor.tflite — neither accelerator produced a usable row on the S26. Both ended the same way: the benchmark process was killed, most likely out of memory.
• kokoro_prosody.tflite — neither accelerator produced a usable row on the S26. NPU: the on-device compile never returned inside the sweep's 1500 s timeout. GPU: the benchmark process was killed, most likely out of memory.
• kokoro_vocoder.tflite — neither accelerator produced a usable row on the S26. NPU: the benchmark process was killed, most likely out of memory. GPU: LiteRtException: Failed to compile model.

file	backend	compiled	inference (median / min)	load
kokoro_82m_fixedlen_fp32.tflite	NPU (Hexagon v81)	on-device JIT	1150.4 ms / 1096.2 ms	3037 ms

Measured on a Samsung Galaxy S26 (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16) with LiteRT CompiledModel 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. The run held thermal status NONE throughout. Headroom 0.67–0.78, where 1.0 is the throttling threshold.

The NPU rows ran the published file unchanged. LiteRT compiled it for the Hexagon on the device at first load. That first compile took 24.3 min here. The load column above is the cached load every later run pays. Recipe and the runtime libraries it needs: NPU guide.

GPU wiring: GPU guide.

Raspberry Pi 5 (CPU)

Measured on a Raspberry Pi 5 Model B Rev 1.1 (8 GB, Raspberry Pi OS 64-bit) with the LiteRT benchmark_model tool from litert-cli-nightly 0.2.0.dev20260805: CPU inference (XNNPACK, 4 threads), 3 invocations per file of 10 warm-up plus 50 timed runs (the tool caps a phase at 150 s, so very slow graphs run fewer — the Runs column is the actual timed total). The latency is the median across invocations; the spread is the min–max over all timed runs. No thermal throttling occurred during these runs (vcgencmd get_throttled stayed 0x0).

File	Inference (median)	Spread (min–max)	Runs	Peak memory
kokoro_82m_fixedlen_fp32.tflite	3,122.2 ms	3,111.7–3,174.8 ms	146	1903 MB
kokoro_predictor.tflite	606.4 ms	602.9–613.6 ms	150	4854 MB
kokoro_prosody.tflite	436.9 ms	429.4–457.6 ms	150	3788 MB
kokoro_vocoder.tflite	11,939.4 ms	11,912.8–12,000.2 ms	39	985 MB

License

Apache-2.0, inherited from hexgrad/Kokoro-82M.