radiant-lyrics/sylla-600m-v1

Sylla-600M v1

Acoustic model for syllable-level lyrics alignment. Sylla powers the AI-timed lyrics in Radiant Lyrics.

• Architecture: Wav2Vec2BertForCTC (w2v-BERT 2.0 Conformer, about 600M parameters), 16 kHz input, 80-bin log-mel features, 20 ms frames.
• Heads: a lead-vocal CTC head (model.safetensors) and a background-vocal head (bg_head.safetensors, linear, same vocabulary).
• Vocabulary: 921 tokens: about 900 characters in native scripts (Latin, kana/kanji, Hangul jamo, Thai, …), plus blank, delimiter and special tokens.
• Trained to hold each letter for as long as it is sung (framewise targets), so syllable ends come straight from the CTC path.
• Base model: mrfakename/w2v-bert-2.0-music-lyrics-ctc (MIT), itself from mrfakename/w2v-bert-2.0-music and facebook/w2v-bert-2.0 (both MIT). See NOTICE.
• Training data: commercial recordings paired with human-timed syllable lyrics. Not included, not distributable.

Results

JamendoLyrics MultiLang (public benchmark)

79 songs in English, Spanish, German and French, scored on word onsets. Sylla aligns from the lyrics text alone, with no timing hints.

System	Correct onsets (<0.3 s) ↑	Average error ↓
Sylla-600M v1 (with the Lockstep aligner)	95.9%	0.22 s
Durand et al. 2023, best multilingual system	94%	0.18 s
Durand et al. 2023, English-trained	89%	0.35 s
Gupta et al. (NUS AutoLyricsAlign)	73%	0.65 s
Base model before fine-tuning (measured by us)	91.0%	0.94 s

Published rows are from Durand et al. 2023, Table 3. Sylla and the base model were measured by us on the current Hugging Face release of the dataset, so the comparison is indicative rather than a controlled reproduction. 92.9% of Sylla's word onsets land within 200 ms.

Syllable timing (held-out songs)

Lead-vocal syllable onsets on songs never seen in training, aligned from the lyrics text alone.

Set	Within 200 ms	Within 300 ms
10 reference-timed songs	89.7%	91.4%
55 Indonesian songs	86.2%	89.3%
Background vocals (same 10 songs)	53.5%

• Versus the base model: average word error falls from 0.94 s to 0.22 s on JamendoLyrics MultiLang, and from 0.92 s to 0.39 s on JamendoLyrics++ (80 English songs, 95.7% correct onsets).
• Speed: about 4 s per song on a single Radeon 7900 XTX.

Input format

Sylla listens to 16 kHz mono audio and the Lockstep aligner times a known lyric against it. Lyrics go in as syllable-timed TTML (W3C Timed Text). The timings in the input can be rough, wrong or missing: only the text and line structure are needed, and every begin / end comes back re-timed.

<tt xmlns="http://www.w3.org/ns/ttml" xmlns:ttm="http://www.w3.org/ns/ttml#metadata"
    xmlns:itunes="http://music.apple.com/lyric-ttml-internal" itunes:timing="Word" xml:lang="en">
  <body>
    <div>
      <p begin="00:12.300" end="00:15.100" itunes:key="L1" ttm:agent="v1">
        <span begin="00:12.300" end="00:12.700">Hel</span><span begin="00:12.700" end="00:13.100">lo</span>
        <span begin="00:13.200" end="00:13.900">world</span>
        <span ttm:role="x-bg"><span begin="00:14.000" end="00:14.600">(world)</span></span>
      </p>
      <p begin="00:16.000" end="00:19.000" itunes:key="L2">A line with only line timing works too</p>
    </div>
  </body>
</tt>

• Lines: one <p> per lyric line, inside <div> sections. itunes:key and ttm:agent are optional and kept as they are.
• Syllables: one <span begin end> per syllable or word. Spans written back to back (Hel + lo) form one word; a space or line break between spans starts a new word.
• Line-only lyrics: a <p> with plain text and no spans works too. It is split into words and each word gets timed.
• Background vocals: wrap their spans in <span ttm:role="x-bg">. They are timed by a separate head.
• Times: mm:ss.mmm, h:mm:ss.mmm, seconds (12.3 or 12.3s) or milliseconds (12300ms).
• Scripts: write lyrics in their native script (kana/kanji, Hangul, …). No romanisation needed.
• Output: the same TTML with every begin / end re-timed and everything else kept. Radiant-AI-CTC also accepts the same lyrics as JSON.

Use

Loaded by Radiant-AI-CTC (RADIANT_MODEL=models/sylla-600m-v1/best):

hf download radiant-lyrics/sylla-600m-v1 --local-dir models/sylla-600m-v1/best

License

Copyright (C) 2026 meowarex

Licensed under the GNU General Public License v3.0 (LICENSE). The base models are MIT; their notices are kept in NOTICE.