RadixArk/Muse-Glimmer-q4km-gs128-MLX

Muse Glimmer — MLX Q4_K_M (gs128)

Overview

An Apple Silicon (MLX) checkpoint of Muse Glimmer, produced as a code-preserving repack of the vendor's llama.cpp GGUF muse-glimmer-30B-kquant-17gb.gguf. Every weight keeps the GGUF's exact quantization code; only the group scales are re-expressed in MLX affine form. Serve it with SGLang's MLX backend on a Mac with 48 GB of unified memory or more. Text only — the MLX backend has no vision path.

Model Specifications

• Source: the vendor GGUF release → gguf_weights/muse-glimmer-30B-kquant-17gb.gguf (sha256 7e9b74b7…), revision aa336138… (the 2026-08-10 "definitive Muse Glimmer 30B GGUFs" re-publish; its quantized payload is byte-identical to the earlier the previous vendor GGUF — this repack's tensors are bit-identical to the previous revision of this repo, so all evaluation results below are unchanged)
• Format: MLX-quantized safetensors, 4 shards (~17 GB); tokenizer / config / chat template from the vendor HF export
• Quantization: Q4_K blocks → 4-bit affine; Q6_K (attn_v, ffn_down) → 6-bit affine group-128 via lossless sub-scale run merge (13,671,424 superblocks hard-asserted, zero violations, no requantization); Q5_K lm_head → 5-bit group-128
• Fidelity: quantization codes bit-identical to the GGUF; recombined scales stored as bf16 (≤2⁻⁸ relative rounding)
• Architecture: 28B dense, 52 layers (13 full-attention NoPE + 39 sliding-window-2048), context 131,072
• Modality: text only

Evaluation Results

Measured through SGLang MLX on an M5 Pro (64 GB), with the vendor llama.cpp fork serving the source GGUF on the same machine as the reference. GSM8K: 200 questions, no-thinking chat template, temperature 0. CIMemories: single trial, DeepSeek-R1-0528 judge (single-trial judge variance applies).

Benchmark	This checkpoint (SGLang MLX)	llama.cpp, same GGUF
GSM8K	0.970	0.970
CIMemories violation (lower better)	0.00%	8.27%
CIMemories coverage (higher better)	76.0%	68.4%

Performance (M5 Pro 64 GB, 1k-in/1k-out greedy)

Batch size	1	2	4	6	8
Decode tok/s (aggregate)	15.3	27.2	46.5	46.8	52.6

Ahead of llama.cpp on the same GGUF codes at every batch size above 1 (llama.cpp: 16.1 → 37.3 tok/s).

Serving with SGLang

SGLANG_USE_MLX=1 SGLANG_MLX_CACHE_LIMIT_GB=8 \
python -m sglang.launch_server \
  --model-path RadixArk/Muse-Glimmer-q4km-gs128-MLX \
  --trust-remote-code \
  --reasoning-parser muse \
  --tool-call-parser muse \
  --disable-radix-cache \
  --mem-fraction-static 0.85 \
  --host 0.0.0.0 --port 30000

SGLANG_MLX_CACHE_LIMIT_GB=8 caps the MLX buffer cache (keeps the footprint within a 48 GB machine at no throughput cost). Keep --disable-radix-cache. See the Muse Glimmer page in the SGLang cookbook for the full deployment matrix.

Sibling checkpoints

• RadixArk/Muse-Glimmer-q4-MLX — fastest MLX variant, quantized from the BF16 HF export
• RadixArk/Muse-Glimmer-q4k-dynamic-MLX — mirror of the vendor's searched dynamic K-quant