Instructions to use stabilityai/stable-audio-3-optimized with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Stable Audio 3
How to use stabilityai/stable-audio-3-optimized with Stable Audio 3:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
cpu-amx README: document the bf16 DiT tier (--dit-precision)
Browse files- cpu-amx/README.md +8 -4
cpu-amx/README.md
CHANGED
|
@@ -15,7 +15,7 @@ Torch-free **C++ / Intel-AMX** inference engines for **Stable Audio 3 medium**,
|
|
| 15 |
pipeline on CPU with no PyTorch, MLX, TFLite, or stable-audio-tools at runtime — just the `.so`s + numpy.
|
| 16 |
|
| 17 |
```
|
| 18 |
-
prompt ─▶ T5Gemma (C++ AMX) ─▶ DiT pingpong (C++ AMX int8) ─▶ SAME-S/L decoder (C++ AMX) ─▶ WAV
|
| 19 |
audio ─▶ SAME-S/L encoder (C++ AMX) ─▶ latent ─▶ … (audio-to-audio / inpainting)
|
| 20 |
```
|
| 21 |
|
|
@@ -35,7 +35,8 @@ than fp32** — rebuild from source (`optimized/cpu-amx/build/`) for other ISAs.
|
|
| 35 |
| file(s) | model | precision / method | notes |
|
| 36 |
|---|---|---|---|
|
| 37 |
| `t5gemma_bf16.*` | T5Gemma-b-b-ul2 encoder (text conditioner) | **bf16** AMX GEMMs, fp32 islands (RMSNorm/softmax/softcap/RoPE) | standard softmax attention → bf16-safe |
|
| 38 |
-
| `dit_medium_int8.*` + `dit_medium_int8_kernels.tar.gz` | medium DiT (24-block rectified-flow) | **int8** — fully-fused all-integer, AOT-compiled Triton kernels + oneDNN | naive RTN (weight-PTQ *hurts* the DiT — see below) |
|
|
|
|
| 39 |
| `same_{s,l}_decoder_bf16.*` | SAME-S (50M) / SAME-L (426M) decoder | **bf16** AMX GEMMs, fp32 differential-attention islands | highest-fidelity CPU decoders |
|
| 40 |
| `same_{s,l}_decoder_int8.*` | SAME-S / SAME-L decoder | **int8 (w8a8, fused)** via **SmoothQuant α0.9 → GPTQ** ("improved" grid) | ½ size; SAME-S +1.2 dB over naive int8 on real music |
|
| 41 |
| `same_{s,l}_encoder_bf16.*` | SAME-S / SAME-L encoder (audio→latent) | **bf16** AMX GEMMs, fp32 differential-attention islands | for audio-to-audio / inpainting |
|
|
@@ -43,8 +44,11 @@ than fp32** — rebuild from source (`optimized/cpu-amx/build/`) for other ISAs.
|
|
| 43 |
| `same_l_encoder_bf16_weights_f32.bin` | SAME-L encoder (optional) | **fp32** refinement mode | max-fidelity; bf16 is the default |
|
| 44 |
|
| 45 |
**Quantization findings baked into these choices** (full write-up in the repo's `LESSONS.md`):
|
| 46 |
-
- The medium **DiT stays naive-int8** — GPTQ/SmoothQuant *lower* its accuracy (its
|
| 47 |
-
the chaotic 8-step sampler make a calibration-averaged Hessian overfit and
|
|
|
|
|
|
|
|
|
|
| 48 |
- The **decoders are activation-limited**: SmoothQuant (activation-outlier migration) does the work,
|
| 49 |
GPTQ just lets α go higher. The gain **transfers to real music for SAME-S (+1.2 dB), not SAME-L**.
|
| 50 |
- **bf16** is used where attention is either standard-softmax (T5Gemma) or where the fp32 differential-
|
|
|
|
| 15 |
pipeline on CPU with no PyTorch, MLX, TFLite, or stable-audio-tools at runtime — just the `.so`s + numpy.
|
| 16 |
|
| 17 |
```
|
| 18 |
+
prompt ─▶ T5Gemma (C++ AMX) ─▶ DiT pingpong (C++ AMX int8 | bf16) ─▶ SAME-S/L decoder (C++ AMX) ─▶ WAV
|
| 19 |
audio ─▶ SAME-S/L encoder (C++ AMX) ─▶ latent ─▶ … (audio-to-audio / inpainting)
|
| 20 |
```
|
| 21 |
|
|
|
|
| 35 |
| file(s) | model | precision / method | notes |
|
| 36 |
|---|---|---|---|
|
| 37 |
| `t5gemma_bf16.*` | T5Gemma-b-b-ul2 encoder (text conditioner) | **bf16** AMX GEMMs, fp32 islands (RMSNorm/softmax/softcap/RoPE) | standard softmax attention → bf16-safe |
|
| 38 |
+
| `dit_medium_int8.*` + `dit_medium_int8_kernels.tar.gz` | medium DiT (24-block rectified-flow) | **int8** — fully-fused all-integer, AOT-compiled Triton kernels + oneDNN | naive RTN (weight-PTQ *hurts* the DiT — see below); default, 1-thread core |
|
| 39 |
+
| `dit_medium_bf16.*` (reuses the int8 kernels tarball) | medium DiT | **bf16** AMX GEMMs (oneDNN, packed weights) + bf16 flash attention, **fp32 RoPE/RMSNorm islands** | **near-lossless** (~59/54 dB @ L1292/L4096, cos 0.9997+) — `--dit-precision bf16`; ~1.24× the int8 latency, runs at `--threads` |
|
| 40 |
| `same_{s,l}_decoder_bf16.*` | SAME-S (50M) / SAME-L (426M) decoder | **bf16** AMX GEMMs, fp32 differential-attention islands | highest-fidelity CPU decoders |
|
| 41 |
| `same_{s,l}_decoder_int8.*` | SAME-S / SAME-L decoder | **int8 (w8a8, fused)** via **SmoothQuant α0.9 → GPTQ** ("improved" grid) | ½ size; SAME-S +1.2 dB over naive int8 on real music |
|
| 42 |
| `same_{s,l}_encoder_bf16.*` | SAME-S / SAME-L encoder (audio→latent) | **bf16** AMX GEMMs, fp32 differential-attention islands | for audio-to-audio / inpainting |
|
|
|
|
| 44 |
| `same_l_encoder_bf16_weights_f32.bin` | SAME-L encoder (optional) | **fp32** refinement mode | max-fidelity; bf16 is the default |
|
| 45 |
|
| 46 |
**Quantization findings baked into these choices** (full write-up in the repo's `LESSONS.md`):
|
| 47 |
+
- The medium **DiT's quantized tier stays naive-int8** — GPTQ/SmoothQuant *lower* its accuracy (its
|
| 48 |
+
adaLN-modulated qkv + the chaotic 8-step sampler make a calibration-averaged Hessian overfit and
|
| 49 |
+
generalize worse than RTN). For fidelity-over-speed there's a **near-lossless `bf16` DiT** tier
|
| 50 |
+
(`--dit-precision bf16`): bf16 only on the AMX matmuls (GEMMs + flash attention), RoPE/RMSNorm kept in
|
| 51 |
+
fp32 islands (bf16-RoPE angle breaks long renders). ~59/54 dB vs the int8 tier's ~40/36 dB, at ~1.24× the latency.
|
| 52 |
- The **decoders are activation-limited**: SmoothQuant (activation-outlier migration) does the work,
|
| 53 |
GPTQ just lets α go higher. The gain **transfers to real music for SAME-S (+1.2 dB), not SAME-L**.
|
| 54 |
- **bf16** is used where attention is either standard-softmax (T5Gemma) or where the fp32 differential-
|