whisper-small-waxal-matched-ewe (matched-recipe Ewe ablation)

Research ablation, not the recommended Ewe model. This is Whisper-Small fine-tuned on Ewe only, with exactly the recipe of the leave-one-out model whisper-small-waxal-loo-ewe. It exists so that "trained on Ewe" and "trained on the 18 other languages" are compared under one recipe. For Ewe ASR use whisper-small-waxal-ewe.

Result (Ewe test split)

Setting	WER	CER
Leave-Ewe-out (18 other languages, same recipe)	101.4	56.8
This model (Ewe only, same recipe)	78.7	53.1
Released per-language Whisper-Small (benchmark recipe)	32.3	n/a

Test utterances scored: 1891.

Training

All Ewe utterances in the WAXAL train split (google/WaxalNLP), 16 kHz mono; transcripts NFC-normalized, lower-cased, punctuation removed, diacritics preserved. Labels longer than Whisper's 448-token limit are truncated.

Hyperparameter	Value
Base	`openai/whisper-small` (244M)
Training languages	1 (Ewe)
Steps	4,000
Optimizer	AdamW, lr 1e-5, 200 warmup steps
Batch size	16
Precision	fp16
Language token	none (same as the leave-one-out model)
Hardware	1x NVIDIA H200

Evaluation

Greedy decoding on the full Ewe WAXAL test split; WER and CER with jiwer on NFC-normalized, lower-cased text with diacritics preserved.

Usage

from transformers import WhisperForConditionalGeneration, WhisperProcessor
import soundfile as sf

model = WhisperForConditionalGeneration.from_pretrained("waxal-benchmarking/whisper-small-waxal-matched-ewe")
processor = WhisperProcessor.from_pretrained("waxal-benchmarking/whisper-small-waxal-matched-ewe")
wav, sr = sf.read("your_audio.wav")            # 16 kHz mono
feats = processor(wav, sampling_rate=16000, return_tensors="pt").input_features
ids = model.generate(feats)
print(processor.batch_decode(ids, skip_special_tokens=True))

Intended and prohibited uses

Intended use. Research on, and development of, speech recognition for the language(s) this model was trained on, in particular for low-resource and under-served language communities. Error rates are still too high for high-stakes uses such as healthcare, legal, or emergency settings without further validation on the target domain.

Prohibited uses. This model must not be used for non-consensual applications, including surveillance or monitoring of individuals or communities, covert recording or transcription of people without their consent, or voice cloning and impersonation. Please respect the consent and privacy of the speakers whose speech you process.

Citation

Part of the WAXAL ASR Benchmark (arXiv:2606.02375).

@article{waxalnet2026,
  title  = {The WAXAL ASR Benchmark: Fine-Tuned Edge Models Across 19 African Languages},
  author = {Olufemi, Victor Tolulope and Babatunde, Oreoluwa and Njema, Ramsey and others},
  year   = {2026},
  note   = {arXiv preprint arXiv:2606.02375}
}