Arm/pocket-tts-fr-24l-mix-precision

Pocket TTS French 24 Large - Mixed Precision and Mixed Frameworks

This repository packages a mixed-precision, mixed-framework deployment bundle derived from kyutai/pocket-tts for French text-to-speech using the Large 24-layer FlowLM variant. The bundle uses SentencePiece, GGUF, ONNX, and LiteRT artifacts and supports streaming synthesis at 24 kHz.

✨ Key Highlights

• Faster than real-time reference — a 24-layer package was measured using one Arm® CPU core with SME2:
  • RTF is 0.582 on an Android™ Vivo X300 smartphone.
• Low-latency reference — measured with the 24-layer package on an Android™ Vivo X300 smartphone:
  • Median end-to-end latency is 1.978 s for a prompt that produces 3.40 seconds of audio.
  • Median time to first audio chunk is 0.398 s.
• Memory-efficient reference — measured with the 24-layer package:
  • Peak process RSS is 2695.2 MiB on an Android™ Vivo X300 smartphone.
• Large deployment — the packaged model files total 1289.8 MB.
• French text-to-speech — generates spoken French from a text prompt.
• Higher-quality Large variant — uses a 24-layer FlowLM for higher-quality synthesis, with slower inference than the smaller variant.
• Packaged voice profiles — includes 26 voice profiles.
• Arm® CPU deployment — optimized for efficient execution on Arm® CPUs.
• Streaming output — returns synthesized speech at 24 kHz in streaming mode.

📦 Model Details

Model Description

Pocket TTS is a lightweight text-to-speech model developed by Kyutai for efficient CPU execution. This package contains the French Large variant with a 24-layer FlowLM.

• Developed by: Kyutai
• Model type: French text-to-speech with a Large 24-layer FlowLM and packaged voice profiles
• License: CC BY 4.0
• Base model: kyutai/pocket-tts
• Package form: SentencePiece, GGUF, ONNX, TFLite, and NumPy artifacts

Model Sources

• Base model: https://huggingface.co/kyutai/pocket-tts
• Upstream repository: https://github.com/kyutai-labs/pocket-tts
• Voice collection: https://huggingface.co/kyutai/tts-voices

🚀 Get Started with the Model

🔓 Compute Flow — Early Access

The inference engine for this model package is available through the Compute Flow Early Access Program.

Want to try it?

📩 Contact us at ai-early-access@arm.com to request access.

📊 Quality evaluation

Accuracy results for this mixed-precision package will be provided in a future update. However, no intelligibility regression was observed against the upstream FP32 implementation.

🎯 Performance evaluation

The benchmark figures below were obtained with the German 24-layer package. The French variant uses the same runtime architecture and tensor dimensions, so similar performance is expected when prompts produce comparable output durations.

Performance was measured under the following conditions:

• One Arm® CPU thread.
• 5 warmups followed by 30 consecutive measured runs with no pause between runs.
• The Android™ Vivo X300 smartphone screen was kept on.
• The model remained loaded and its state was reset between runs.

The following methodology and definitions were used:

• Runtime: LiteRT, llama.cpp, ONNX Runtime, and SentencePiece on CPU, with XNNPACK and KleidiAI.
• Input prompt: "Hallo zusammen. Ich bin Jack und ich bin Ihr persönlicher Assistent."
• Voice profile: juergen, the German package default.
• Generated output: 3.40 seconds of 24 kHz audio per run.
• End-to-end latency is the summed model execution time for the complete utterance and excludes model setup.
• Time to first audio chunk is the elapsed execution time until the first streaming audio chunk is returned.
• Average memory is the mean process RSS sampled throughout setup and inference.
• Peak memory is the maximum sampled process high-water mark.
• Latency and memory were collected in separate executions to prevent memory sampling from affecting latency.

Metric	German 24-layer reference: Android™ Vivo X300
Model size	1289.8 MB
RTF	0.582
End-to-end latency, p50	1.978 s
End-to-end latency, p90	2.047 s
End-to-end latency, p99	2.097 s
Time to first audio chunk	0.398 s
Peak memory	2695.2 MiB
Average memory	2659.2 MiB

RTF is the total inference time divided by the generated audio duration, so lower values indicate faster processing and values below 1 indicate faster-than-real-time generation.

🛠️ Technical Specifications

Objective

Generate streaming 24 kHz French speech from a text prompt.

Runtime Architecture

Component role	Framework / format
Text tokenization	SentencePiece
Text encoding, projection, and denoising	ONNX Runtime / ONNX
24-layer flow language modeling	llama.cpp / GGUF
Audio decoding	LiteRT / TFLite
Voice conditioning	NumPy profile

Precision and Quantization

The FlowLM and ONNX components use FP32 weights. The LiteRT audio decoder uses FP16 fully connected weights.

Input Specification

Input	Description
Text prompt	UTF-8 string

Output Specification

The model returns synthesized 24 kHz audio in streaming mode.

Repository Contents

• pocket_tts_manifest.json — model package manifest.
• tokenizer.model — SentencePiece tokenizer.
• *.onnx, *.gguf, and *.tflite — model components.
• assets/*.npy — packaged voice profiles and model assets.
• metadata.yaml — model metadata.
• SHA256SUMS — model-bundle checksums for reproducibility.

🗂️ Model and Asset Origin

• Pocket TTS model
• Pocket TTS source repository
• Kyutai TTS voice collection and asset licensing

🔐 Checksums

SHA256SUMS was generated by recursively hashing every regular file in the model bundle, including files in subdirectories, except the generated root SHA256SUMS and paths with a dotfile component.

From the model bundle root, verify the checked-out files with:

shasum -a 256 -c SHA256SUMS