Qwen3.8-Flash-Next-Abliterated-MTPLX-4bit

Abliterated 4-bit build of Qwen3.8-Flash-Next optimized for MTPLX

Updated 2026-08-31 — please re-download

If you downloaded this model before 2026-08-31, you have v1. Re-download for roughly a 10% speed boost.

v2 is forged directly from the abliterated BF16 weights rather than repackaged from the oQ4e MLX build. Same size, same architecture, same sidecars.

	v1 Original	v2 (live version)	delta
Recommended: MTP, draft_depth 1	38.0 – 61.6 tok/s	41.1 – 67.4 tok/s	+8-9%
Short, thinking xhigh, temp 1, MTP	45.1	47.9	+6.2%
Long agentic (26.5k), thinking xhigh, temp 1, MTP	38.7	43.6	+12.7%
Fastest single cell: short, temp 0, MTP	61.6 (thinking low)	67.4 (thinking medium)	+9.4%
Short, thinking xhigh, temp 1, AR	34.9	48.2	+38.1%
Long agentic, thinking xhigh, temp 1, AR	31.0	43.5	+40.3%
Size on disk	115.7 GB	116.0 GB	+0.3%

Measured on the same Mac Studio M3 Ultra, same prompts, same harness, both on MTPLX 2.10.1. v2 wins all 41 comparable cells across three fixtures, both modes, all thinking levels, both temperatures.

Lineage: Qwen/Qwen3.8-Flash-Next → Blackfrost-AI/Qwen3.8-Flash-Next-DERISKED-BF16 (abliterated) → this build, forged for MTPLX.

Abliteration (upstream BF16, Qwentium persona)	2.3% refusal on harmful prompts (7/300), 2.2% full suite (10/450)

Intended use and terms

Published for experimental AI research and AI-safety research only: red-teaming, refusal and alignment research, interpretability, and quantization or inference research.

It must not be used for anything illegal, in any jurisdiction, under any circumstances. The removal of refusal behavior is a research property of the artifact, not an invitation, a permission, or an endorsement. Nothing about publishing this model condones or excuses illegal activity.

You are solely responsible for your use of it and for complying with applicable laws and upstream licence terms. It ships with essentially no safety refusals: assume it will comply with any instruction, including harmful ones. Run it locally on hardware you control. Do not expose it as a public endpoint or deploy it to untrusted users.

---

Will it fit on my Mac?

There are two parts to load: the model itself (84.0 GB) and a 32.0 GB n-gram table that helps it run faster. The model always sits in memory. The n-gram table is the part you have a choice about.

	memory needed
n-gram table kept in memory (faster)	~116 GB
n-gram table read from SSD	~84 GB

You do not have to configure this. On a Mac with 160 GB or more of unified memory, MTPLX keeps the n-gram table in memory by itself. On any Mac with less than that, it reads the table from your SSD instead and the model still runs normally, roughly 12% slower.

To override the automatic choice, MTPLX_NGRAM_RESIDENT=1 pins the n-gram table in memory and MTPLX_NGRAM_RESIDENT=0 pins it to SSD. Unset means automatic, so on a large Mac you need =0 to stream.

Quick start

Apple Silicon. Resident n-gram wants ~116 GB of unified memory; streaming the table from SSD instead needs ~84 GB plus a hot cache.

pip install mtplx

hf download grant-ai/Qwen3.8-Flash-Next-Abliterated-MTPLX-4bit \
  --local-dir ~/models/qwen38-abliterated-mtplx

MTPLX_NGRAM_RESIDENT=1 mtplx serve \
  --model ~/models/qwen38-abliterated-mtplx \
  --model-id qwen38-abliterated \
  --generation-mode mtp --depth 1 \
  --port 8000

On a Mac with less than ~160 GB of RAM, drop MTPLX_NGRAM_RESIDENT=1. MTPLX streams the n-gram table from SSD automatically below that threshold and the model still runs, about 12% slower.

Then use the OpenAI-compatible endpoint:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model": "qwen38-abliterated",
       "messages": [{"role": "user", "content": "Explain MoE routing."}],
       "max_tokens": 1024, "temperature": 1.0, "top_p": 0.95, "top_k": 20}'

--generation-mode mtp --depth 1 is the recommended configuration. It is substantially faster than ar at temperature 0 and at short context. At long context with temperature 1 the two are level. See Speed below.

Speed

Hardware: Mac Studio M3 Ultra, 32-core CPU, 80-core GPU, 256 GB unified memory, fans pinned to max. MTPLX 2.10.1 stock, measured 2026-08-31 at --depth 1. Decode throughput, engine-reported, median of 2 reps at temperature 0 and 3 at temperature 1.

Short context (289 tokens)

AR baseline: 42.9 to 44.5 tok/s at temp 0, 48.2 to 49.8 tok/s at temp 1.

thinking level	decode tok/s (temp 0)	gain over AR	decode tok/s (temp 1)	gain over AR
off	59.9	+34.6%	54.9	+13.9%
low	66.0	+53.5%	54.8	+10.0%
medium	67.4	+56.4%	59.5	+22.4%
xhigh	60.6	+41.3%	47.9	-0.6%

Long agentic context (26,531 tokens)

AR baseline: 38.8 to 39.3 tok/s at temp 0, 43.4 to 43.5 tok/s at temp 1.

thinking level	decode tok/s (temp 0)	gain over AR	decode tok/s (temp 1)	gain over AR
off	43.4	+10.4%	41.1	-5.5%
low	43.1	+10.8%	42.5	-2.1%
medium	43.9	+12.6%	42.6	-1.8%
xhigh	43.4	+11.9%	43.6	+0.2%

At long context the speed does not depend on how hard the model thinks. All eight cells land between 41.1 and 43.9 tok/s. Reasoning at xhigh over 26k tokens of context costs nothing against thinking off.

Speculative decoding pays at temperature 0 and at short context. At long context with temperature 1 it is level with plain autoregressive decoding: the baseline is fast enough that drafting has little left to win.

Draft depth

Measured at long agentic context, thinking xhigh, temperature 1.

draft depth	decode tok/s	tokens per draft step
1	42.3	1.53
2	40.9	1.79
3	37.6	1.81
4	32.1	1.76
5	28.2	1.77
6	refused by the engine

Deeper drafting commits more tokens per step and is still slower: the extra draft work costs more time than the extra accepted tokens return.

Vision

Vision works. The 333 vision tensors ship at fp16 in this pack and the tower is constructed by MTPLX's serving path, not by the text backend. Re-verified on 2.10.1 with a live image request.

Requires --generation-mode mtp. In ar mode the server refuses image content outright. That is a clean refusal, not a silent failure.

Abliteration

Abliteration has not been remeasured on this build. The figures are upstream BF16, carried through quantization unchanged.

refusal rate, harmful prompts	2.3% (7 of 300)
refusal rate, full suite	2.2% (10 of 450)
persona measured	Qwentium

Personas

Ships with the stock Qwen chat template as the default, which injects no system prompt. The abliterated weights carry the behaviour on their own.

The upstream Blackfrost "Qwentium" persona is included as an opt-in: a ~1,200 token compliance-oriented system prompt prepended to every request. Switch with the bundled script, then restart your server:

bash switch_persona.sh blackfrost   # persona on
bash switch_persona.sh qwen         # back to stock (default)
bash switch_persona.sh status       # which is active

The template lives in two places, chat_template.jinja and the chat_template key inside tokenizer_config.json, and different runtimes read different ones, so the script switches both. It is safe against an hf download cache checkout: it replaces symlinks rather than writing through them.

Settings that matter

setting	value
sampler	temperature 1.0, top_p 0.95, top_k 20 (from the model's own generation_config.json)
generation mode	--generation-mode mtp --depth 1
n-gram residency	MTPLX_NGRAM_RESIDENT=1. Auto-arms at >=160 GB RAM
QSA gather	MTPLX_QSA_GATHER=1 MTPLX_QSA_GATHER_DECODE=1 MTPLX_QSA_GATHER_MAX_ROWS=16 adds about 2% at long context and costs about 1% at short. Off by default in MTPLX 2.10.1
MTP depth	1. Depth 3 never wins; depths 4 and 5 are slower than not speculating
thinking	chat_template_kwargs: {"enable_thinking": false} to disable
reasoning effort	--reasoning-effort low | medium | xhigh, or reasoning_effort in the API

Technical details

architecture	qwen4_exp, hybrid Gated-DeltaNet + Qwen Sparse Attention, MoE, PLE n-gram embeddings
quantization	4-bit group 32 base; 8-bit group 64 on embed_tokens, lm_head, linear_attn.out_proj, mlp.gate, shared_expert_gate, shared_expert.{gate,up,down}_proj and indexer.index_qk_proj; hyper-connection mixers and ple.{key,value}_proj kept bf16
bit widths	4 and 8 bit, group sizes 32 and 64
MTP head	13 tensors quantized: 4-bit g32 on the switch_mlp and self_attn projections, 8-bit g64 on the shared-expert and indexer projections. Shipped as mtp.safetensors (1.68 GB)
vision	333 tensors, fp16, working in mtp mode
runtime	MTPLX 2.10.1 (stock)
n-gram table	320,001,536 rows, 4-bit g32, shipped as ngram-table.safetensors (32.0 GB)

How it was made

1. Forged from Blackfrost-AI/Qwen3.8-Flash-Next-DERISKED-BF16 through mtplx forge build on MTPLX 2.10.1, using the Optimized-Speed recipe above. Unlike v1, the body is genuinely requantized rather than passed through from an existing MLX build.
2. mlx-lm ships no qwen4_exp backend, so a shim exposing MTPLX's own implementation was needed for the convert to load the model at all.
3. The MTP sidecar was re-sanitized: forge extracts it carrying the source's fused mlp.experts.* tensor names, which the MTP module rejects.
4. The MTP head was quantized to the reference layout, 5.21 GB down to 1.68 GB.
5. The n-gram table was quantized to 4-bit group 32 into ngram-table.safetensors. The bf16 forge lane otherwise leaves it inline at 102.4 GB.
6. Packaged from a tree containing only the shards the weight index references.

Steps 2 through 6 are not reproducible from mtplx forge build alone on MTPLX 2.10.1. Rebuilding this artifact from the recipe requires them.

Not measured: output quality was not compared against v1. The two builds use different quantization recipes, and only speed was benchmarked.