kylegregory/wav2vec2-large-xlsr-53-ceb

Wav2Vec2 Cebuano / Bisaya ASR

This repository contains a fine-tuned Wav2Vec2 CTC automatic speech recognition (ASR) model for Cebuano / Bisaya speech transcription. The model is built on top of facebook/wav2vec2-large-xlsr-53 and uses a grapheme-level tokenizer designed for Cebuano/Bisaya transcription.

Model summary

• Base model: facebook/wav2vec2-large-xlsr-53
• Architecture: Wav2Vec2 for CTC
• Task: Automatic Speech Recognition
• Primary language: Cebuano / Bisaya
• Tokenizer: grapheme-based CTC tokenizer
• Word delimiter: |
• Training setup: supervised fine-tuning on a curated Cebuano/Bisaya speech dataset
• Intended use: research, prototyping, low-resource ASR experimentation, transcription bootstrapping, and labeling-loop workflows

This repo contains the acoustic model and processor.
If you use KenLM-assisted decoding, that requires an external language model and decoder configuration not bundled in this repository unless explicitly added.

---

What this model does

The model transcribes spoken Cebuano/Bisaya audio into text.

It is designed for:

• Cebuano/Bisaya speech transcription
• low-resource ASR experiments
• semi-automatic transcription and correction workflows
• dataset bootstrapping for iterative labeling
• benchmarking greedy, beam, and LM-assisted decoding strategies

It is not a fully general multilingual ASR model and should not be treated as one.

---

Intended use

Recommended use cases

• academic research
• low-resource ASR experimentation
• internal transcription pipelines
• transcription prefill for human correction
• labeling-loop systems where users review and correct model output

Out-of-scope use cases

• high-stakes transcription without human verification
• legal, medical, or safety-critical transcription
• speaker identification
• emotion recognition
• robust transcription in extremely noisy or adversarial environments
• language varieties far outside the model’s training distribution

---

Training details

This model was fine-tuned from facebook/wav2vec2-large-xlsr-53 using the CTC objective on a curated Cebuano/Bisaya speech dataset.

Acoustic model configuration

• Base model: facebook/wav2vec2-large-xlsr-53
• Processor/tokenizer: custom grapheme tokenizer
• Training objective: CTC
• Feature encoder: frozen during baseline fine-tuning
• Training device: CPU-based training in the reported baseline runs
• Learning rate: 1e-4
• Warmup steps: 500
• Train batch size: 2
• Eval batch size: 2
• Gradient accumulation: 8
• Max epochs: 60
• Checkpoint interval: every 2000 steps

Tokenization

This model uses a grapheme-level tokenizer with:

• lowercase character vocabulary
• | as the word delimiter
• [UNK] for unknown tokens
• [PAD] for padding

Audio expectations

Best results are expected when audio is:

• WAV format
• mono
• 16 kHz
• reasonably clean speech
• short-form utterances rather than long-form broadcast audio

---

Evaluation summary

Reported metrics below are from the project’s current evaluation snapshot.

Checkpoint-6000 evaluation

Decoder	WER	CER	Avg forward-pass time (s/clip)
Greedy	0.0550	0.0159	0.2983
Beam	0.0562	0.0160	0.2983
Beam + KenLM	0.0396	0.0123	0.2983

Interpretation

• The acoustic model alone is already producing strong baseline results on the evaluated test split.
• In this evaluation snapshot, beam search without KenLM did not improve over greedy decoding.
• Beam + KenLM produced the best results, reducing both WER and CER.
• The forward-pass time shown is model inference only. It does not include full end-to-end serving overhead, web latency, upload time, or full beam search pipeline latency.

Important note

The Beam + KenLM results depend on an external language model and decoder setup. Those improvements should not be interpreted as coming from the acoustic model alone unless the language model artifacts are also provided and used.

---

Known limitations

This model has important limitations.

Dataset scope

• Training data size is still modest compared with large-scale commercial ASR systems.
• Some prompts and speaking conditions may be relatively narrow in style.
• Performance may vary across speakers, accents, and recording devices.

Language coverage

• The model is focused on Cebuano / Bisaya.
• It may degrade on:
  • heavy code-switching
  • strongly mixed English-Cebuano speech
  • dialectal variants not well represented in training
  • proper nouns and rare words

Acoustic robustness

• Performance may degrade under:
  • background noise
  • reverberation
  • overlapping speech
  • distant microphone capture
  • music-heavy recordings

Decoding behavior

• Greedy, beam, and KenLM-assisted decoding can produce different outputs.
• External language models can improve average performance but may also introduce bias toward more common word sequences.

---

Bias, risks, and safety considerations

This is a research model for low-resource ASR. Errors are expected.

Potential issues include:

• speaker-dependent performance variation
• poorer transcription of rare names, borrowed words, or uncommon vocabulary
• reduced robustness for unseen accents or recording conditions
• decoding bias introduced by external language models
• transcription errors that could distort meaning in downstream workflows

Human review is strongly recommended before using outputs for publication, decision-making, or dataset release.

---

How to use

Install

pip install transformers torchaudio torch

### Load the model

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

repo_id = "kylegregory/wav2vec2-cebuano"

processor = Wav2Vec2Processor.from_pretrained(repo_id)
model = Wav2Vec2ForCTC.from_pretrained(repo_id)

### Basic Inference Example

import torch
import torchaudio
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

repo_id = "kylegregory/wav2vec2-cebuano"

processor = Wav2Vec2Processor.from_pretrained(repo_id)
model = Wav2Vec2ForCTC.from_pretrained(repo_id)

waveform, sr = torchaudio.load("sample.wav")

# Convert to mono if needed
if waveform.shape[0] > 1:
    waveform = waveform.mean(dim=0, keepdim=True)

# Resample to 16 kHz if needed
target_sr = 16000
if sr != target_sr:
    waveform = torchaudio.functional.resample(waveform, sr, target_sr)
    sr = target_sr

inputs = processor(
    waveform.squeeze().numpy(),
    sampling_rate=sr,
    return_tensors="pt",
    padding=True,
)

with torch.no_grad():
    logits = model(**inputs).logits

pred_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(pred_ids)[0]

print(transcription)