abiawilliamsa/Wav2Vec-AKAN-ASR-0.323

Wav2Vec-XLS-5-AKAN-ASR

Speech-to-text for Akan (Twi), fine-tuned from facebook/wav2vec2-xls-r-300m with a CTC head on Paywinful/akan_asr_dataset.

Metric	Value
WER	0.3249
CER	0.1093

Measured on the full validation split at the best checkpoint (step 3000, epoch 3.66), selected by load_best_model_at_end on WER.

Usage

import librosa, torch
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

processor = Wav2Vec2Processor.from_pretrained("abiawilliamsa/Wav2Vec-XLS-5-AKAN-ASR")
model = Wav2Vec2ForCTC.from_pretrained("abiawilliamsa/Wav2Vec-XLS-5-AKAN-ASR").eval()

speech, _ = librosa.load("clip.wav", sr=16000, mono=True)
inputs = processor(speech, sampling_rate=16000, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

print(processor.batch_decode(logits.argmax(dim=-1))[0])

Audio must be mono 16 kHz — the feature extractor does not resample for you.

Output format

Greedy CTC decoding over a 31-character vocabulary: a–z, the Akan vowels ɔ and ɛ, plus | (word delimiter), [UNK] and [PAD] (the CTC blank). Output is lowercase and carries no punctuation, no capitalisation and no digits — the training transcripts were stripped of all three.

There is no language model attached. Adding a KenLM n-gram decoder over Akan text would likely cut WER further without retraining.

Training

Base model	facebook/wav2vec2-xls-r-300m (300M params)
Data	13,100 train clips / full validation split
Learning rate	3e-4, linear schedule, warmup_ratio=0.1
Batch	4 per device × 2 accumulation steps = effective 8
Precision	fp16
SpecAugment	mask_time_prob=0.075
Feature encoder	frozen
Seed	42

Trained for 3.66 epochs (3000 steps) before the session cap; the run was resumable from checkpoints and best-checkpoint selection means the released weights are the best-WER state, not merely the last.

Evaluation history

Step	Epoch	WER	CER	Eval loss
250	0.31	1.0000	1.0000	5.681
500	0.61	0.7129	0.2609	1.925
1000	1.22	0.4178	0.1416	1.065
1500	1.83	0.3861	0.1294	0.935
2000	2.44	0.3566	0.1191	0.862
2500	3.05	0.3424	0.1141	0.838
3000	3.66	0.3249	0.1093	0.807

WER was still falling at the cap, so further epochs should help.

Two earlier runs on a 10% slice for reference: LR 3e-4 reached WER 0.516, and LR 5e-5 collapsed to all-blank CTC output (WER 1.0). Learning rate was the dominant factor.

Known limitations

• Digits. Training transcripts had digits deleted (afe 1990 mu → afe mu) while the speaker still pronounced them, so the model saw audio with no matching label there. Expect weak behaviour on spoken numbers.
• Domain. Only evaluated on this corpus's read-speech style. Spontaneous, noisy or telephone audio is untested.
• Dialect. The corpus is Twi-dominant; other Akan varieties (Fante, Akuapem) are not separately evaluated.
• Greedy decoding only — no beam search, no language model.

Demo

Interactive Gradio demo: abiawilliamsa/akan-asr-demo