Alpa (अल्प) — Sanskrit speech in 18 MB

Alpa: Sanskrit speech in 18 MB. An English voice model converted to Sanskrit with two tables edited by hand and 100 minutes on a CPU, no GPU. Model size 18 MB, 4.5 million parameters. GPU time used: 0 hours. Phonemes misread: 3.1%, against 3.3% for human recordings. Speed in a browser tab: 12 times faster than real time on one thread.

An English voice model, converted to Sanskrit: two tables edited by hand, then 100 minutes on a CPU. No GPU. v0.2 adds a second voice after two more CPU hours.

Early preview, v0.2. Two trained voices, 9.8 hours of speech, checked by one listener and one recogniser. See Limits.

New in v0.2: a female voice, and a male voice that learned from twice the recordings. See What changed in v0.2.

Technical report: alpa.blog/report tells how it was made, with every measurement and the mistakes along the way. It has a button that copies everything your own AI agent needs to try the same for another language.

Listen

Text male female
धर्मक्षेत्रे कुरुक्षेत्रे समवेता युयुत्सवः। मामकाः पाण्डवाश्चैव किमकुर्वत सञ्जय॥
नटः पटं धरति, बालः पत्रं पठति। कटिः कति? दण्डः दन्तः च।
घटे जलं नास्ति। कोटिः जनाः नाटकं पश्यन्ति। गुडः मधुरः, पीडा कठिना।
यथा धर्मः तथा अर्थः। बुद्धिमान् भीमः सुखेन कथां कथयति।
विद्या ददाति विनयं, विनयाद् याति पात्रताम्।

Each sample was generated once; nothing was re-rolled or selected. Verse is read aloud as text; Alpa does not chant.

What changed in v0.2

v0.2 has new weights and a second voice. Still no GPU.

v0.1.1 v0.2
Voices trained on Sanskrit male male and female
Speech it learned from 3.5 hours 9.8 hours (7.1 male, 2.7 female)
Training in total, on a four-core CPU 100 minutes 3 hours 39 minutes
  • The female voice is new. Until now it was tiny-voca's English voice carried along; our listener judged it not good enough and heard it fade at the visarga. Training only that voice's own 384 numbers (26 minutes) was too little. Training the whole model on both voices (93 minutes) did it: her visarga now has the loudness it has in the recordings (2.0 dB under the vowel before it; recordings 2.1 dB; before 4.7 dB), and the recogniser hears her ट correctly between vowels 94% of the time (before 76%).
  • The male voice learned from twice the recordings. The recogniser now hears his ट between vowels correctly in 21 cases of 21 (before 18). His overall count is a little higher than in v0.1.1 (3.2% of phonemes misread against 3.0%), and one line of short phrases is heard worse; see Limits.
  • On side-by-sides of both voices, our listener judged v0.2 clearly better.

The earlier files are still here: alpa-v0.1-preview.pt with onnx/alpa-v0.1.1.onnx is v0.1.1.

What changed in v0.1.1

v0.1.1 kept the weights of v0.1. What changed is how the model's output is turned into sound, and nothing was trained for it. v0.2 speaks the same way.

Audio
v0.1
v0.1.1

श्रीः स्त्री शास्त्रं स्पष्टं पश्यति। क्षत्रियः राष्ट्रं रक्षति।

After the release, listeners reported a buzz. Our listener narrowed it down: in श, ष and क्ष, and then in ज, घ and ध, it sounded like "two metallic people saying at once". The cause was in the inherited sound generator. It makes the voice from pitched pulses, and the model had learnt to build its hiss, breath and bursts out of those same pulses, so every noisy sound carried a second, pitched voice inside it.

v0.1.1 corrects this while speaking:

  • Each sound gets a frequency limit. Below it the generator is fed the voice's pulses; above it, plain noise of the same loudness. Sibilants and च are noise throughout; stops, ज and ह keep the voice only below 1 kHz; vowels keep it up to 4.5 kHz, as real voices do.
  • Sharper vowels. The resonances that tell one vowel from another were flatter than in the recordings; they are restored to the recordings' level.
  • Smaller touches: a steadier pitch, less whisper inside vowels, and a slightly brighter top.

Measured against the recordings, the noisy sounds now have no pitch above 1 kHz, as in real speech, and the recogniser's score is unchanged (3.0% of phonemes misread, 3.1% before). On a side-by-side of six sentences our listener judged the new voice clearly better. --plain gives the earlier sound.

From English to Sanskrit, without a GPU

Three steps. 1: start with tiny-voca, an English voice of 4.5 million parameters. 2: rewrite two tables by hand, 0.2% of its numbers, with no training; a Sanskrit recogniser then misreads 26.6% of phonemes. 3: 100 minutes on a CPU with 3.5 hours of Sanskrit speech; it then misreads 3.1%, against 3.3% for human recordings.

tiny-voca is a 4.5-million-parameter model that speaks English. Its author trained it from scratch in 7.7 hours on an RTX 4090. Alpa is that same network, converted to Sanskrit in two moves.

1. Rewrite two small tables by hand. Of the model's 4.5 million numbers, 9,600 (0.2%) say which sound is which and how heavy a syllable is. We replaced those, by lookup, with no training:

  • each Sanskrit sound starts from the nearest English sound (आ from the vowel of "father", क from "k");
  • ट ड ण ष, which English lacks, start as 70% of their nearest English sound plus 30% of English "r", because a retroflex and an English "r" are both made with the tongue curled back;
  • laghu and guru take the place of unstressed and stressed.

The other 99.8% of the network was copied untouched. At that point, with no training at all, it already read Sanskrit with an English accent.

2. Give it 100 minutes on a four-core CPU, with 3.5 hours of Sanskrit speech to listen to. Preparing those recordings took about another hour on the same CPU. That was v0.1; the figures and pictures in this section are v0.1's.

Stage Training Phonemes misread by a Sanskrit recogniser
English model with the two tables rewritten none 26.6%
The same, after 100 minutes on a CPU 3,000 steps 3.1%
The human recordings it learned from 3.3%

Starting from the English weights is what makes this cheap. In an early side-by-side on the same 20 minutes of audio and the same 400 steps, a model starting from random weights misread 46.8% and the edited English model 18.3% (measured then with a different, English-leaning recogniser).

The edit that training could not replace

English has no ट, ड or ण, and training alone did not find them. We ran the same 3,000 steps twice, once with those four sounds starting as plain copies of their English neighbours and once with the "30% r" start. That is a difference of 768 numbers.

For each of ट, ड and ण, the share heard correctly between two vowels rises from the plain start to the hand-edited start: ट from 24% to 86%, ड from 52% to 81%, ण from 83% to 97%. Human recordings score 100%, 90% and 100%.

ट ड ण between two vowels, heard correctly ट ड ण
Plain start, 3,000 steps 24% 52% 83%
Hand-edited start, 3,000 steps (this model) 86% 81% 97%
Human recordings 100% 90% 100%

What the recogniser wrote for नटः पटं धरति … कोटिः जनाः नाटकं पश्यन्ति:

  • plain start: नतः पतं धरति … कोतिः जनाः नातकं पश्यन्ति
  • hand-edited start: नटः पटं धरति … कोटिः जनाः नाटकं पश्यन्ति

Small

Download sizes drawn to one scale: Alpa 18 MB, the Vāgdhenu browser port 296 MB, Vāgdhenu 1.8 GB, Indic Parler-TTS 3.75 GB.

Model What it does Parameters Download Size against Alpa
Alpa v0.1 Sanskrit speech 4.5 M 18 MB 1×
Vāgdhenu, browser port Sanskrit chant quantised 296 MB 16×
Vāgdhenu Sanskrit chant 337 M + 112 M vocoder 1.8 GB 99×
Indic Parler-TTS Speech in many Indian languages, Sanskrit among them 938 M 3.75 GB 206×

Sizes are the files on each model's Hugging Face page on 16 November 2026. The larger models are larger for good reasons: Vāgdhenu chants in metre, and Indic Parler-TTS covers many languages and voices. Alpa does one thing, in a size that fits in a web page.

Runs in a browser tab

The whole path, from typed Devanagari to audio, runs inside the page: the text reader under Pyodide, the model under ONNX Runtime Web on one thread. Nothing is sent to a server.

Speed in a browser tab on one thread: 12.4 times real time as measured, 2.9 times with the CPU slowed fourfold, 1.9 times with it slowed sixfold. All three are faster than real time.

Measured in Chromium, one thread Speed A 7.8-second verse takes
As is 12.4× real time 0.7 s
CPU slowed 4× 2.9× real time 2.7 s
CPU slowed 6× 1.9× real time 4.3 s

These timings are of the v0.1 file. The v0.1.1 file, measured beside it on the same day, is about 5% slower (14.0× against 14.5× real time on that day's machine); the v0.2 file is the same graph with new weights.

The model file for the browser is 23 MB; with both runtimes a page needs 47 MB in total.

The only other in-browser Sanskrit voice we know of is the port of Vāgdhenu, a chant system. By its own published benchmark it downloads 296 MB and, on a Pixel 10 Pro XL with six threads, needs about 15.5 seconds to chant a 7.5-second verse, with the first quarter-verse arriving after about 3.8 seconds.

These two are not the same test. Theirs is a real flagship phone; ours is a desktop browser on a 2.1 GHz cloud CPU, slowed with Chrome's own CPU throttle to imitate slower hardware. Alpa has not yet been timed on a real phone.

The file is onnx/alpa-v0.2.onnx; onnx/alpa-v0.2.json lists its inputs and the phoneme table. Its voice input is 0 for female and 1 for male. The earlier files take the same inputs and are still here.

<script src="https://cdn.jsdelivr.net/npm/onnxruntime-web@1.20.1/dist/ort.wasm.min.js"></script>
<script type="module">
  // ids and weight come from alpa/text.py, which runs unchanged under Pyodide
  ort.env.wasm.wasmPaths = "https://cdn.jsdelivr.net/npm/onnxruntime-web@1.20.1/dist/";
  const int64 = (v) => new ort.Tensor("int64", BigInt64Array.from(v.map(BigInt)), [1, v.length]);
  const one = (x) => new ort.Tensor("float32", Float32Array.from([x]), [1]);
  ort.env.wasm.numThreads = 1;
  const session = await ort.InferenceSession.create("alpa-v0.2.onnx", { executionProviders: ["wasm"] });
  const out = await session.run({ tokens: int64(ids), weight: int64(weight),
      voice: new ort.Tensor("int64", BigInt64Array.from([1n]), [1]),      // 1 = male
      speed: one(1), pitch_shift: one(0), pitch_range: one(1) });
  // out.audio.data is a Float32Array at 24 kHz
</script>

Scores

The judge is Su-śrotā, a Sanskrit speech recogniser. It listened to sentences held out of training, 148 for the male voice and 145 for the female, spoken once by Alpa from text alone and once by the original voice artist.

Alpa v0.2, male Male recordings Alpa v0.2, female Female recordings
Phonemes misread 3.2% 3.3% 3.0% 3.3%
ट heard as ट 93% 97% 94% 100%
ड heard as ड 89% 94% 97% 100%
ण heard as ण 100% 99% 100% 100%

v0.1.1's male voice scored 3.0%, 94%, 87% and 97%.

Read these with two cautions. The recogniser knows Sanskrit words and partly guesses from context, which flatters every column; the stricter between-vowels figures for v0.1 are in the table further up. And a recogniser is not an ear: one human listener has checked a handful of lines.

Against other systems. There is no shared benchmark for Sanskrit speech synthesis yet, and the other systems have not been run through this judge, so we make no quality claim against them. Vāgdhenu's authors report a mean opinion score of about 4.6 from an expert listener, for chant. Alpa has no listener-panel score. On desktop speed the published figures are: Alpa about 100 times faster than real time on four CPU threads; Vāgdhenu about 3.5 times faster than real time on an M5 Max.

Use it

pip install "torch>=2.4" numpy soundfile huggingface_hub
hf download 77ethers/alpa --local-dir alpa-model
cd alpa-model
python -m alpa.infer --checkpoint alpa-v0.2.pt --voice female \
    --text "सत्यमेव जयते नानृतम्।" --output out.wav

From Python, inside the same folder:

import soundfile as sf
from alpa.infer import load_model, speak

model = load_model("alpa-v0.2.pt")
audio, phonemes, _ = speak(model, "सत्यमेव जयते नानृतम्।", "male")     # or "female"
sf.write("out.wav", audio.numpy(), 24000)

speak also takes speed, pitch_shift and pitch_range. To hear the model without the corrections of v0.1.1, add --plain to the command or pass polish=None to speak. python -m alpa.export writes the browser file (it needs pip install onnx onnxruntime). Text in Kannada, Telugu, Bengali or Malayalam script is read as well if you pip install indic-transliteration. The model file holds plain tensors and settings only and is loaded with weights_only=True.

Parameters 4,531,723
Files alpa-v0.2.pt (18 MB), onnx/alpa-v0.2.onnx (23 MB). Earlier: alpa-v0.1-preview.pt, onnx/alpa-v0.1.1.onnx, onnx/alpa-v0.1-preview.onnx
Code alpa/: the text reader, the conversion from tiny-voca, data preparation, the trainer, the speaker and the exporter. scripts/: four helpers. TRAINING.md: every command, in order
Audio 24 kHz, mono
Input Sanskrit in Devanagari
Voices male and female, both trained on Sanskrit
Licence Apache-2.0

Make your own

Everything that made Alpa is here. TRAINING.md lists the commands in the order they were run, from tiny-voca's English weights to alpa-v0.2.pt, with how long each took on four CPU cores, and ends with the six files you would change for another language. The technical report gives the reasons and the measurements, and has a button that copies a brief for your own AI coding agent.

What the text reader does

  • Every written vowel is spoken, including the final a: राम is rāma.
  • An aspirated stop is spoken as the stop followed by ह.
  • Anusvāra takes the class of the stop that follows it inside a word.
  • Comma, daṇḍa, double daṇḍa, question mark and exclamation mark become pauses.
  • Each syllable is marked laghu or guru, and the model is told which.
  • The text is read as written. Sandhi is neither applied nor undone.

Limits

  • The female voice has had less training than the male. It learned from 2.7 hours, he from 7.1. The corrections of v0.1.1 were tuned on the male voice and sharpen her vowels more than her recordings have.
  • Short, fast phrases can lose a sound. In असतो मा सद्गमय। तमसो मा ज्योतिर्गमय। the male voice's त of असतो and म of तमसो are faint, and ॐ on its own is often not heard. Speaking a little slower (speed=0.9) helps.
  • Little data and a short run. It learned from 9.8 hours, and the adversarial stage of tiny-voca's recipe, which adds much of the naturalness, has not been run. The voice can sound flatter than larger systems.
  • The cleaner voice is a correction, not a cure. Since v0.1.1 the sound is fixed while speaking; the model itself still makes the faults, and --plain lets you hear them.
  • A few vowels come out hoarse. The ए of ते in कर्मण्येवाधिकारस्ते… मा ते सङ्गोऽस्त्वकर्मणि (male voice) is the known case; it is not fixed.
  • ण and न are not yet fully distinct to a human listener.
  • Word-final visarga is sometimes weak. The recogniser missed it in three of the five samples above, in each voice.
  • It speaks; it does not chant. Vedic accent marks are ignored.
  • Untested: single words on their own, very long compounds, long passages, real phones. One call takes up to 512 symbols; split longer text at sentence marks.
  • Digits and Latin letters are not spoken.

Credits and licence

Alpa is released under the Apache License 2.0. It stands on other people's work:

  • tiny-voca by Muhammed Nazeem (Apache-2.0): the model design, the training recipe, the browser export and the starting weights. Alpa's weights are derived from tiny-voca's. See THIRD_PARTY_NOTICES.md.
  • Rasa by AI4Bharat (CC-BY-4.0): the recordings it learned from.
  • Vāgdhenu and Su-śrotā by Prathosh A P, Indian Institute of Science: the pronunciation and scansion rules follow Vāgdhenu's, and Su-śrotā was used to measure the result. Neither is included here.
  • wav2vec2 phoneme recogniser (facebook/wav2vec2-lv-60-espeak-cv-ft): used to time the sounds in the training recordings.

This is an independent project. None of the people or groups above has reviewed or endorsed it.

@inproceedings{ai4bharat2024rasa,
  author={Praveen Srinivasa Varadhan and Ashwin Sankar and Giri Raju and Mitesh M. Khapra},
  title={{Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settings}},
  year=2024,
  booktitle={Proc. INTERSPEECH 2024},
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for 77ethers/alpa

Base model

n4ze3m/tiny-voca
Finetuned
(3)
this model

Dataset used to train 77ethers/alpa

Space using 77ethers/alpa 1