Qwen3-4B-Instruct β€” L5 dialect (Pure expression) Β· reward ablation gr3b110

A LoRA adapter that makes Qwen/Qwen3-4B-Instruct-2507 reason at compression level L5 β€” a single collapsed expression.

This is an ablation, not one of the headline models: it is the same level trained under a different reward, published so the reward-design comparison in the paper can be rerun rather than taken on faith. For the main model at this level see ssurface/cot-dialect-qwen3-4b-instruct-grpo-l5.

Results

Accuracy
This adapter 74.3%

GSM8K test (n=1317), greedy decoding, single-turn, no exemplars, no self-consistency.

Training data

GSM8K train, re-expressed at level L5 by a teacher model: 6993 examples, median chain length 16 characters inside <think>.

Across the family the median chain runs from 532 characters at L1 to 16 at L5 β€” a 33x span. An L5 chain looks like this:

18/3*2=12

Training setup

GRPO on top of the merged level-5 SFT model.

Engine trl.GRPOTrainer on stock transformers, attention sdpa
Reward correctness, format, chain, gr3
Loss type dapo
Generations per prompt 8
Batch 16 x 2 accum
Max completion 256 tokens
Learning rate 1e-05
KL coefficient (beta) 0.0
Prompt set gsm8k_grpo_balanced_1k.json
Trained on merged_new_fixed/l5
LoRA r=16, alpha=32
Hardware 1x NVIDIA A100 80GB

Reward components

  • correctness β€” +/- the gold solution's step count on an answer match, so harder problems are worth more
  • format β€” the response must be one <think>...</think> block then #### <answer>
  • chain β€” a verifier: the arithmetic written inside the chain must actually check out
  • gr3 β€” multiplicative length rescaling of the positive combined reward, floored at 0.3 -- it scales rewards that are already positive, so it cannot reorder correct above incorrect

Engine note. Stock transformers with sdpa attention, not a fused-kernel wrapper. The fused path produced adapters whose lora_B matrices were all zero β€” mathematically inert despite loading without error. Every adapter in this collection was verified lora_B != 0 before publishing; 13 that failed that check were withheld.

Usage

Solve this using Level 5 (Extreme).
Problem: {your problem}

Stacks on the SFT model, not the raw base. Trained against the merged SFT model, so loading it straight onto Qwen/Qwen3-4B-Instruct-2507 will not reproduce the number above.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "ssurface/cot-dialect-qwen3-4b-instruct-sft-l5")   # 1. SFT for this level
model = model.merge_and_unload()
model = PeftModel.from_pretrained(model, "ssurface/cot-dialect-qwen3-4b-instruct-grpo-gr3b110-l5")   # 2. this adapter
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")

Limitations

  • Trained and evaluated on math word problems only.
  • Accuracy falls with problem difficulty, fastest at the compressed levels.
  • Single seed unless the repo name says otherwise; differences of a couple of points are within noise (95% half-width ~2.7 pp at n=1317, ~4.4 pp at n=500).
  • Ablation artefact. It was trained to answer one question about reward design and may be worse than the core model at the same level.

Citation

@misc{cot-compression-dialects,
  title  = {Chain-of-Thought Compression Dialects},
  author = {Frolov, Anatolii},
  year   = {2026}
}
Downloads last month
13
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for ssurface/cot-dialect-qwen3-4b-instruct-grpo-gr3b110-l5

Adapter
(5,861)
this model

Dataset used to train ssurface/cot-dialect-qwen3-4b-instruct-grpo-gr3b110-l5

Collection including ssurface/cot-dialect-qwen3-4b-instruct-grpo-gr3b110-l5

Evaluation results