Orpheus 3B 0.1 ft — GGUF (ggml-quantised)

GGUF / ggml conversion of canopylabs/orpheus-3b-0.1-ft (sourced via the non-gated unsloth/orpheus-3b-0.1-ft mirror) for use with CrispStrobe/CrispASR.

Orpheus 3B is a Llama-3.2-3B-Instruct talker finetuned to emit <custom_token_N> codec tokens that the SNAC 24 kHz codec decodes back to speech. Key capabilities include human-like speech with natural intonation and emotion, zero-shot voice cloning, guided emotion/intonation via simple tags, and low-latency streaming (~200 ms). 8 fixed English speakers (tara, leah, jess, leo, dan, mia, zac, zoe).

Pair this with the SNAC codec at cstr/snac-24khz-GGUF — the talker outputs codec tokens but doesn't render audio without it.

Files

File	Quant	Size	Notes
`orpheus-3b-0.1-ft-f16.gguf`	F16	6.16 GB	Reference quality
`orpheus-3b-0.1-ft-q8_0.gguf`	Q8_0	3.70 GB	Recommended — ASR roundtrip word-exact vs F16
`orpheus-3b-0.1-ft-q4_k.gguf`	Q4_K	2.38 GB	Smallest; smoke-tested but Q8_0 preferred
`diff-harness-ref/orpheus-talker-ref.gguf`	—	614 KB	Diff-harness reference checkpoint (not for inference)

The talker LM is sensitive to peaked codec distributions. Sub-Q8 quants tend to break the SNAC super-frame slot pattern and produce gibberish even when the LM perplexity remains plausible; Q4_K is provided for constrained-memory setups but Q8_0 is the safe default.

Quick start

# 1. Build CrispASR
git clone https://github.com/CrispStrobe/CrispASR
cd CrispASR
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --target crispasr-lib

# 2. Pull the talker + the SNAC codec
huggingface-cli download cstr/orpheus-3b-0.1-ft-GGUF orpheus-3b-0.1-ft-q8_0.gguf --local-dir .
huggingface-cli download cstr/snac-24khz-GGUF snac-24khz.gguf --local-dir .

# 3. Synthesise
./build/bin/crispasr --backend orpheus \
    -m orpheus-3b-0.1-ft-q8_0.gguf \
    --codec-model snac-24khz.gguf \
    --voice tara \
    --temperature 0.6 \
    --tts "Hello, my name is Tara." \
    --tts-output hello.wav

24 kHz mono WAV. --voice <name> picks one of the 8 baked speakers; --temperature 0.6 is the upstream engine_class.py default and is required — greedy decoding (--temperature 0) enters a 7-slot loop after a few super-frames and produces unusable audio.

For auto-download simply pass -m auto:

./build/bin/crispasr --backend orpheus -m auto \
    --voice leo --temperature 0.6 \
    --tts "Auto-download fetches both files." \
    --tts-output out.wav

Architecture

Component	Details
Talker LM	Llama-3.2-3B-Instruct (28 layers, 3072 hidden, 24 heads, 8 KV heads, head_dim=128, vocab 128256 + 7×4096 codec tokens)
RoPE	NEOX, theta=500000
Codec	hubertsiuzdak/snac_24khz (RVQ, 3 codebooks × 4096) — separate GGUF
Sampling	temperature=0.6 + top-k by default; greedy is unstable
Audio	24 kHz mono float32 PCM

The talker emits a stream of <custom_token_N> LM tokens; every 7 emitted tokens form one "super-frame" that de-interleaves into 1 codes_0 / 2 codes_1 / 4 codes_2 entries (per orpheus_tts_pypi/orpheus_tts/decoder.py). 4 super-frames cover 16 SNAC frames (× 512-sample hop = 8192 PCM samples at 24 kHz).

Prompt format (verbatim from canopyai/Orpheus-TTS)

[audio_start=128259, BOS=128000, ...tokenize("{name}: {text}")...,
 eot_id=128009, audio_eot=128260, audio_eom=128261, audio_end=128257]

The Llama-3 BOS at position 1 is critical. Without it the talker still emits well-structured super-frames but the audio is semantically garbage. The CrispASR runtime handles this for you — direct callers of orpheus_synthesize_codes need to mirror the layout.

Stop policy

Stop on audio_end=128257 or on >4 consecutive non-codec tokens. Don't stop on audio_pre_end=128009 or audio_end_b=128261 — those overlap with Llama-3 specials in the prompt and text_N<10 reserved markers in the custom_token block; the upstream tokens_decoder filters them silently rather than terminating on them.

Conversion

python models/convert-orpheus-to-gguf.py \
    --input unsloth/orpheus-3b-0.1-ft \
    --output orpheus-3b-0.1-ft-f16.gguf \
    --outtype f16

build/bin/crispasr-quantize orpheus-3b-0.1-ft-f16.gguf orpheus-3b-0.1-ft-q8_0.gguf q8_0
build/bin/crispasr-quantize orpheus-3b-0.1-ft-f16.gguf orpheus-3b-0.1-ft-q4_k.gguf q4_k

Drop-in checkpoint variants

The orpheus runtime is checkpoint-agnostic — same arch, same prompt format, same SNAC codec. GGUF mirrors of:

cstr/kartoffel-orpheus-3b-german-natural-GGUF (German, natural speech, 19 speakers)
cstr/kartoffel-orpheus-3b-german-synthetic-GGUF (German, synthetic data, 26 speakers)

are checkpoint swaps. They all reuse the same SNAC codec at cstr/snac-24khz-GGUF.

Attribution

Talker base model: canopylabs/orpheus-3b-0.1-ft (Apache-2.0). canopylabs / canopyai.
Non-gated mirror used for conversion: unsloth/orpheus-3b-0.1-ft.
Llama base: meta-llama/Llama-3.2-3B-Instruct — Llama-3.2 community license.
SNAC codec: hubertsiuzdak/snac_24khz (MIT) — see cstr/snac-24khz-GGUF.
Reference TTS engine: canopyai/Orpheus-TTS (engine_class.py:_format_prompt, decoder.py).
GGUF conversion + ggml runtime: CrispStrobe/CrispASR — see src/orpheus.cpp, src/orpheus_snac.cpp, models/convert-orpheus-to-gguf.py.

License

Apache-2.0 (as declared by canopylabs/orpheus-3b-0.1-ft). The underlying Llama-3.2-3B-Instruct weights carry the Llama-3.2 community license with its Acceptable Use Policy and "Built with Llama" attribution requirement — review both before redistribution.

The SNAC codec is MIT and ships separately under cstr/snac-24khz-GGUF.

Provenance and EU AI Act Art. 53 note

Upstream model: canopylabs/orpheus-3b-0.1-ft — published by canopylabs.
Upstream licence: apache-2.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
What was done here: format conversion and/or quantisation only (GGUF/GGML). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository. No training-content summary was found on the upstream model card at the time of writing; that documentation gap is upstream's and is not filled here.
Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.