saryps-labs/pocket-tts-hindi

Pocket TTS Hindi

v0.1 — early research release

Pocket TTS Hindi is a compact, streaming Hindi text-to-speech model with prompt-based voice cloning. It uses the six-layer Pocket TTS runtime architecture and contains 109.5M parameters. The release weights use BF16 storage and occupy 219 MB.

This model was produced in two stages: a 24-layer Hindi teacher was trained from scratch, then distilled into a six-layer student while classifier-free guidance was baked into a single inference pass.

Highlights

• Hindi speech synthesis and voice cloning
• 109,502,146 parameters
• 219 MB BF16 weight file
• streaming generation
• faster than real-time on the tested x86 CPU
• optional dynamic-int8 CPU execution
• 24 kHz mono output

This is a public research checkpoint, not a production safety-certified service.

Quick start

The current release requires a Pocket TTS checkout containing support for append_terminal_punctuation: false in custom configs. From that environment:

uv run pocket-tts generate \
  --config hf://saryps-labs/pocket-tts-hindi/config.yaml \
  --voice ./your_hindi_voice.wav \
  --text "आज का दिन बहुत अच्छा है" \
  --output-path hindi_output.wav

Dynamic int8 CPU inference:

uv run pocket-tts generate \
  --config hf://saryps-labs/pocket-tts-hindi/config.yaml \
  --voice ./your_hindi_voice.wav \
  --text "आज का दिन बहुत अच्छा है" \
  --output-path hindi_output_int8.wav \
  --quantize

Native INT8 ONNX runtime

The onnx/ directory contains a reviewed CPU-oriented ONNX conversion and a PocketTTS.cpp source snapshot for native/offline use without PyTorch. The five graphs are self-contained ONNX opset 17 models. Saryps Labs independently verified graph integrity, native Linux generation, BOS voice conditioning, and basic Hindi output; the contributor also tested Windows and NVDA integration. The approximately 240 MB bundle uses selective dynamic INT8 quantization: main-transformer MLP projections, the flow network, and decoder are quantized, while sensitive main-transformer attention projections remain FP32.

Easiest native ONNX setup

Prerequisites: Python 3, CMake 3.28+, a C++17 compiler, and Git. Download the model repository, then use the included helper. It builds the native runtime automatically on the first invocation and reuses it afterward:

pip install -U huggingface_hub
hf download saryps-labs/pocket-tts-hindi \
  --include "onnx/*" \
  --local-dir pocket-tts-hindi

python pocket-tts-hindi/onnx/run_onnx.py \
  --voice ./your_hindi_voice.wav \
  --text "आज का दिन बहुत अच्छा है" \
  --output hindi-output.wav

The voice prompt should be a clear WAV or FLAC recording from a speaker who has consented to voice cloning. The first run downloads native build dependencies and can take several minutes; later generations start directly.

See onnx/README.md for build and generation instructions and onnx/VALIDATION.md for detailed provenance, benchmarks, and limitations.

Use a clear voice prompt from a speaker who has consented to cloning. The current model was trained without meaningful punctuation coverage, so plain Hindi text without punctuation is recommended.

Model architecture

Component	Configuration
FlowLM transformer	6 layers, width 1024, 16 heads
Flow head	depth 6, width 512
Audio codec	Mimi, 12.5 latent frames/s
Output sample rate	24 kHz
Tokenizer	Hindi SentencePiece, 4,000 tokens
Total parameters	109,502,146
Storage dtype	BF16
Weight size	219,029,196 bytes

The student was distilled for 200,000 optimizer steps from a 24-layer teacher trained for 400,000 steps. CFG coefficient 2.0 was distilled into the student, which runs with one conditioned backbone pass at inference.

Training data

The training manifest contains 469,795 utterances and 932.11 hours:

Dataset	Utterances	Hours
AI4Bharat IndicVoices Hindi	381,496	803.31
AI4Bharat Kathbath Hindi	88,299	128.81

Separate pools contain 13.49 validation hours and 11.37 speaker-disjoint evaluation hours. Audio/text pairs were filtered, normalized, and aligned with a Hindi wav2vec2 CTC aligner. Dataset licenses and attribution are documented in NOTICE.md.

Evaluation

A deterministic 100-item cross-sentence protocol used different prompt and target utterances from the same held-out speaker:

• 50 IndicVoices and 50 Kathbath cases
• 21 held-out speakers
• temperature 0.3
• one flow decode step
• EOS threshold -1.0
• seed 20260901

Intelligibility and reliability

Metric	Generated	Genuine held-out audio
Whisper large-v3 WER	23.28%	26.74%
Whisper large-v3 CER	11.54%	13.89%
WavLM speaker similarity	0.9305	0.9292
UTMOS	3.1234	2.5469

• silent outputs: 0/100
• maximum-length/no-EOS failures: 1/100

Automatic metrics are evaluator-dependent. Better ASR scores than genuine references can reflect cleaner or more canonical synthesized speech and do not prove perfect human-perceived quality.

CPU benchmark

AMD EPYC 7V13 Azure VM, one PyTorch thread, three Hindi sentences, three measured repetitions after warm-up:

Metric	FP32 runtime	Dynamic int8
Median speed	3.39× real-time	3.74× real-time
Median time to first audio	110 ms	72 ms
Peak process RSS	1.32 GiB	1.20 GiB

Int8 preserved intelligibility on the same 100-item protocol (10.66% CER and 22.32% WER), with the same one no-EOS case and zero silent generations. Hardware results should not be generalized to every edge device.

Sample generations

All samples are synthetic outputs from held-out voice prompts. Prompt audio is not redistributed in this release.

Voice	Sentence	Audio
Female-labelled 1	आज सुबह मौसम बहुत सुहावना था और बच्चे मैदान में खेल रहे थे	Listen
Female-labelled 2	भारत में अनेक भाषाएँ बोली जाती हैं और हर भाषा की अपनी सुंदर परंपरा है	Listen
Female-labelled 3	आज सुबह मौसम बहुत सुहावना था और बच्चे मैदान में खेल रहे थे	Listen
Male-labelled 1	भारत में अनेक भाषाएँ बोली जाती हैं और हर भाषा की अपनी सुंदर परंपरा है	Listen
Male-labelled 2	आज सुबह मौसम बहुत सुहावना था और बच्चे मैदान में खेल रहे थे	Listen
Male-labelled 3	भारत में अनेक भाषाएँ बोली जाती हैं और हर भाषा की अपनी सुंदर परंपरा है	Listen

Limitations

• Hindi only; code-switching and English words were not systematically tested.
• Punctuation was effectively absent from training. The release config disables automatic terminal punctuation, and punctuation-controlled prosody is not supported reliably.
• One of 100 held-out evaluation cases failed to emit EOS before the maximum generation length.
• Some prompts can transfer recording noise into generated speech. Female samples occasionally exhibited reduced crispness during informal listening.
• Fully quantizing every supported main-transformer MatMul produced a robotic word-level artifact during ONNX testing. The distributed ONNX profile avoids that configuration and uses the selectively quantized profile that passed the final human smoke review.
• Numbers, abbreviations, rare names, dialects, accents, and long-form text need broader human evaluation.
• WavLM, Whisper, and UTMOS scores are model-dependent proxies.
• Consumer-laptop and low-power edge benchmarks are pending.

Intended use and safety

Intended for Hindi TTS research, local/offline applications, accessibility experiments, and evaluation of compact speech synthesis.

Only clone voices with informed permission. Clearly disclose synthetic speech. Do not use this model for impersonation, fraud, harassment, deceptive media, or rights violations. Users are responsible for complying with applicable law, dataset terms, and voice/publicity rights.

Licenses and attribution

The Hindi model weights are released under CC BY 4.0. Pocket TTS and its Mimi implementation/weights used by this release are provided under the MIT terms identified in NOTICE.md. IndicVoices declares CC BY 4.0; the pinned Kathbath card declares CC BY 4.0 in its metadata and separately describes its IndicSUPERB packaging as CC0. See the notice for pinned revisions, component-level attribution, and citation information.

Collaboration and deployment

We are open to research and commercial collaborations involving deployment, device-specific optimization, and compact custom TTS training for particular voices, domains, or languages. If you are interested in using this model in a product, deploying it on-device, or developing a custom model, contact Saryps Labs through this model's Community tab.

Acknowledgements

Built with Kyutai Pocket TTS. Training speech was provided by AI4Bharat IndicVoices and Kathbath. We thank Kyutai and AI4Bharat for making their work available to the research community. The native ONNX conversion and Windows/NVDA validation were contributed by Jim Grimsby Jr., maintainer of Sonata Sherpa Voices.