Nimruz/Mizan-v3-3B

Mizan v3 · 3B · میزان

by Nimruz · GGUF: Mizan-v3-3B-GGUF · all models: Mizan collection

A Persian-first decision model. You give it a record (text or JSON) and a set of typed questions. It returns a calibrated probability for every allowed answer, in one forward pass. There is no text generation, no parsing and no made-up labels.

• Applies your rules. Put your routing policy or scoring rubric in the record and Mizan v3 follows it, even when the rule goes against intuition.
• Level with models 3–9× its size on 3,000 real Persian records: 82.5%, against 83.0% for Clef-Flash 9B and 83.4% for Clef 27B. Best calibrated of all models tested.
• 91.1 on Tasmim-Bench, ahead of Clef, Jev and Clef-Flash, with the lowest score error on that leaderboard (see the note below).
• Small and fast: it decides from layer 20 of Qwen3.5-4B's 32. The 1.9 GB Q4 GGUF answers in about 55 ms on an entry-level GPU, 1.5× faster than Mizan 1.0.
• Jev / SystemOne compatible: the same request and response body as /v1/systemone.
• Apache-2.0, trained only on data that allows commercial use.

Which Mizan should I use?

Model	Reads	Download (4-bit)	Use it for
Mizan v3 3B (this one)	text, JSON	1.9 GB	The default: routing, policies, scores, moderation, reviews, intent
Mizan v2 4B	text + images	2.7 GB + 0.37 GB	Camera frames, receipts, factory and product photos

• v3 has no vision. If your records include pictures, use v2. It also runs the live demo.
• Mizan 1.0 and 1.5 are older text models. They're kept as the v1 and v1.5 tags of this repo, but v3 replaces both.

What it's good at

Use	Example question
Ticket routing by your policy	"Following the policy in the record, which team gets this?"
Severity, urgency, risk on your rubric	"Impact 0–3: 3 = over 200 people or more than 50M toman lost"
Next step by priority rules	"Close / ask for details / escalate / assign, per the policy"
Fact checks against the record	"The customer gave a tracking number." (yes/no)
Moderation, sentiment, intent, topic	the classic Persian tasks, on reviews, tweets, messages

Questions can be in Persian or English, and they can be new questions it has never seen. Options can have meaningful keys (billing) or opaque ones (opt_1), because it reads the descriptions.

Getting started (5 minutes)

1. Install llama.cpp (build b11443 or newer):

brew install llama.cpp            # macOS / Linux with Homebrew
# or: download a release for your OS from https://github.com/ggml-org/llama.cpp/releases
llama-server --version            # check the build number

2. Download Mizan (1.9 GB):

pip install -U huggingface_hub
hf download Nimruz/Mizan-v3-3B-GGUF mizan-v3-3b-Q4_K_M.gguf --local-dir .

3. Start the server:

llama-server -m mizan-v3-3b-Q4_K_M.gguf -c 8192 -b 8192 -ub 8192 -ngl 99 --port 8080   # GPU
llama-server -m mizan-v3-3b-Q4_K_M.gguf -c 8192 -b 8192 -ub 8192 -ngl 0  --port 8080   # CPU only

4. Ask your first question. A request has a state (your record, as text or JSON) and questions (each with a type and instructions):

import httpx

request = {
    "state": {"review": "غذا سرد رسید و پیک هم خیلی بداخلاق بود"},
    "questions": {
        "sentiment": {"type": "choice", "instructions": "حس کلی مشتری چیست؟",
                      "criteria": {"positive": "راضی", "negative": "ناراضی"}},
        "refund": {"type": "noul", "instructions": "آیا مشتری درخواست بازگشت پول کرده؟"},
    },
}
answers = httpx.post("http://localhost:8080/v1/systemone", json=request, timeout=60).json()["answers"]
print(answers)

The real output of Mizan v3 Q4:

{"sentiment": {"type": "choice", "choice": "negative", "confidence": 0.989,
               "probabilities": {"negative": 0.989, "positive": 0.011}},
 "refund":    {"type": "noul", "noul": 0.040}}

5. Read the answer. There are three question types:

type	You give	You get back
choice	criteria: {key: description}	choice (the top key), confidence, probabilities for every key
noul	a yes/no statement or question	noul: the probability of "yes"
score	criteria: a list of levels, lowest first	score: the expected level, e.g. 2.3; probabilities per level; legend

For a whole-number level, take the level with the highest probability. score is the probability-weighted average, which is useful for sorting.

6. Give it your rules. This is where v3 shines. Put the policy and the rubric in the record:

{
  "state": {
    "message": "سلام، از دیروز اینترنت کل ساختمون ما قطعه، حدود ۴۰ واحد. کد پیگیری K48213",
    "customer": {"tier": "ویژه", "previous_follow_ups": 1},
    "policy": "ارجاع: قطعی و کندی ← شبکه؛ قبض ← صورتحساب. اگر شماره پیگیری نباشد، اول اطلاعات تکمیلی بخواه."
  },
  "questions": {
    "route": {"type": "choice", "instructions": "طبق قواعد، این مورد به کدام تیم برود؟",
              "criteria": {"network": "شبکه", "billing": "صورتحساب", "sales": "فروش"}},
    "impact": {"type": "score", "instructions": "شدت پیامد کدام سطح است؟ بالاترین سطح صادق را انتخاب کن.",
               "criteria": ["فقط یک نفر", "۲ تا ۱۰ نفر", "۱۱ تا ۱۰۰ نفر", "بیش از ۱۰۰ نفر"]},
    "has_reference": {"type": "noul", "instructions": "کاربر شماره پیگیری داده است."}
  }
}

Mizan v3 Q4 answers:

• route network (95.6%);
• impact level 2, "11 to 100 people" (99.6%);
• has_reference yes (97.7%).

Writing good questions

• Rules go in state as plain text. v3 follows the policy it's given, even when the policy is unusual.
• Describe every option. The descriptions matter more than the keys, so opt_1 works as well as billing.
• Give score levels concrete thresholds ("more than 10M toman"), not just "low / high".
• Add an "other / none of these" option when the answer might not be listed.
• Many questions per request is fine, and it costs almost nothing extra: all answers come from one pass.
• Persian or English questions both work, on Persian or English records.

Going to production

• Gate on confidence: auto-apply answers above a threshold and send the rest to a person. On our benchmark, the most confident 70% of answers are about 92% correct.
• Test first on 200–300 of your own labeled records, and pick the threshold from that.
• Throughput: llama-server --parallel N serves several requests at once. We haven't benchmarked it for Mizan yet, so load-test it on your hardware.
• Q8 vs Q4: Q8 (3.1 GB) is about 0.6 points more accurate; Q4 (1.9 GB) is smaller and runs on any 8 GB GPU or a laptop CPU.

راهنمای سریع فارسی

میزان یک مدل تصمیم‌گیری است، نه یک چت‌بات. یک «رکورد» (متن یا JSON) و چند سؤال تایپ‌شده می‌گیرد و برای هر گزینه احتمال برمی‌گرداند:

• choice یعنی انتخاب یک گزینه؛
• noul یعنی بله/خیر؛
• score یعنی سطح‌بندی.

قوانین و معیارهای کسب‌وکارتان را داخل خود رکورد بنویسید. میزان v3 طبق همان قوانین تصمیم می‌گیرد. برای تصویر از میزان v2 استفاده کنید. پیش از خودکارسازی، مدل را روی چند صد نمونهٔ واقعی خودتان بسنجید.

Python without a server

Unsloth (calibrated, same answers as the GGUF):

from unsloth import FastDecisionModel

model, processor = FastDecisionModel.from_pretrained("Nimruz/Mizan-v3-3B", max_seq_length=4096)
FastDecisionModel.for_inference(model)
answers = FastDecisionModel.predict(model, processor, state, questions)

Tested with unsloth==2026.10.3.

Plain transformers:

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("Nimruz/Mizan-v3-3B")
sys.path.insert(0, path)
from joint_schema_model import load_release_model, systemone

model, processor = load_release_model(path, device="cuda")
print(systemone(model, processor, {"model": "mizan", "state": state, "questions": questions}))

Plain transformers gives uncalibrated probabilities: the same top answer, but sharper confidences. To calibrate, divide each question's logits by the temperature in unsloth_decision_config.json for its type (order: choice, score, noul).

Evaluation

1. Real Persian data

3,000 real, human-labeled records from 7 public Persian datasets. Every model gets identical rows and questions, and none of these rows was used for training.

Task	v3 Q8	v3 Q4	Mizan 1.0	Clef-Flash 9B	Clef 27B	Jev (API)	Laya 322M
Food-review sentiment (SnappFood)	87.0	87.0	87.0	88.0	85.8	86.0	80.8
Product recommend (Digikala)	71.8	72.2	74.8	76.2	75.2	74.8	44.5
Aspect question in Persian (ParsiNLU)	81.0	80.8	71.5	80.0	78.5	80.0	67.8
Offensive moderation (Pars-OFF)	86.0	83.8	80.8	82.2	86.8	87.5	72.2
Intent routing, 60 intents (MASSIVE)	77.5	76.8	72.5	87.7	85.7	78.2	27.2
Article topic routing (Digikala Mag)	81.2	80.5	80.8	77.0	81.0	82.2	62.5
Claim supported by passage (FarsTail)	93.0	92.2	83.5	90.0	90.8	97.5	81.8
Mean	82.5	81.9	78.7	83.0	83.4	83.7	62.4
Calibration error (ECE, lower is better)	0.058	0.061	0.110	0.078	0.075	0.092	0.210
Size	3.1 GB	1.9 GB	9.4 GB	~19 GB	~55 GB	hosted	0.7 GB

Earlier releases score 82.0 (1.5) and 82.8 (v2, which adds images). This benchmark is close to saturated: the top models all land at 82–84%, and about half of the "errors" we reviewed on SnappFood and Digikala were wrong dataset labels.

2. Tasmim-Bench (external)

Tasmim-Bench v1.0.4 has 96 synthetic Persian cases: routing, next steps and severity scores under fictional policies, with colloquial text, Finglish, negation and taarof.

	Choice accuracy	Yes/no F1	Score error (MAE) ↓	Overall
Mizan v3 Q4	0.917	0.970	0.458	91.1
Mizan 1.0 Q4	0.870	0.970	0.490	89.2
gpt-6-luna-decisions (API, #1)	0.922	1.000	0.479	92.1
Clef (API)	0.958	0.979	0.729	89.8
Jev 1.13 (API)	0.943	0.989	0.760	89.3

Note: v3 never saw any Tasmim case, and the benchmark's domains were kept out of its training data. But part of that data borrows Tasmim's question format: a fictional policy in the record, 0–3 impact levels, a next-step question. So part of the gain is familiarity with the format. With 96 cases, differences of 1–2 points are within noise.

3. Policies and rubrics (in-house)

267 cases from a domain held out of training (school administration), built by the same generator as v3's rule data. Every answer is computed by code from the case's facts and its policy.

	Overall	Routing by policy	Next step	Impact / urgency scores
Mizan v3 Q4	97.1%	97%	98%	98–99%
Mizan 1.0 Q4	70.5%	–	–	–

The domain is unseen, but the format isn't, so treat this as "it learned the skill", not as a real-world score.

Speed

Same 4-bit GGUF on llama.cpp, NVIDIA L4: 55 ms median per text request, against 80 ms for Mizan 1.0. CPU speed is not measured for v3 yet; expect about the same 1.5× gain.

Confidence gating: auto-decide what Mizan is sure about, and send the rest to a human.

How v3 is built

A shorter backbone. We measured how well each layer of Qwen3.5-4B supports Persian decisions with a linear probe per layer:

Layer	8	12	16	18	20	24	28	32 (last)
Held-out mix	64.7	71.8	78.2	80.3	79.9	78.5	77.6	76.1
Unseen domain	53.5	61.9	72.5	75.7	73.1	72.4	70.6	66.4

• Decision ability peaks in the middle of the model, and the last layers, which specialize in predicting the next word, are worse. Earlier classification work found the same (Chip-Tuning, TELL-TALE).
• So v3 keeps the first 20 of 32 layers, about 2.9B parameters, and puts the decision head there.
• It's still a standard Qwen3.5 checkpoint with fewer layers, so llama.cpp and Unsloth load it unchanged.

Order-aware scores. Score questions get an extra loss term: the expected distance from the right level. Being off by one level costs less than being off by three.

Training

Base	Qwen/Qwen3.5-4B (via unsloth/Qwen3.5-4B), cut to its first 20 decoder layers
Head	Clef joint-schema head, trained from scratch
Method	Unsloth FastDecisionModel + DecisionTrainer; LoRA r=64 on all linear layers (lr 1e-4), head lr 3e-4, ordinal loss weight 0.5; LoRA merged
Schedule	1 epoch, 1,336 steps, batch 16, cosine, max length 4,096
Calibration	Per-type temperatures fitted on a held-out slice (ECE 0.027 there)
Compute	1× H100, 34 min

Data: 21.4k decisions, about 57% English and 43% Persian.

• English: Unsloth's decision mixture (BANKING77, CLINC150, MNLI, SNLI, WANLI, BoolQ, AG News, SST-5, MMLU, CommonsenseQA, ARC, prompt injections) plus typed-decisions.
• Rule-labelled Persian cases (2,530 records, about 11k questions) in 9 business domains: food delivery, banking apps, telecom, insurance, HR, clinic admin, logistics, building management and ride hailing.
  • Code samples the facts and a random policy for each case, then computes every answer. An LLM only writes the Persian message.
  • A second, blind LLM pass reads the facts back from the message, and mismatches are dropped (87% kept).
• Mistake fixes (2,420): we found earlier Mizan models' mistakes on real Persian data, and an LLM diagnosed each one blind. For every real mistake it wrote 3 new examples: new wording, a contrast pair, and a new question wording. All of them are new text, and any example sharing 6 or more consecutive words with the source data was dropped.
• Persian business workflows (about 6,600 records) from Mizan 1.0: moderation, reviews, app feedback, support routing, SMS fraud, listings, sales leads, chatbot guardrails, RAG relevance and CV screening.
• Clean data: no text from datasets with non-commercial or unclear licenses. The only real rows used are from MASSIVE (CC BY 4.0).
• Leak control: records and their copies are split together, and training rows are decontaminated against every eval set.

Limitations

• Text only. For images, use Mizan v2.
• Most Persian training data is synthetic. Test it on 200–300 of your own labeled records before automating.
• Weaker than Clef on large intent taxonomies (60 intents: 77.5% vs 87.7%), and slightly below the earlier Mizan models on Digikala's "recommend or not" reviews.
• The 4-bit build costs about 0.6 points against 8-bit; use Q8 when accuracy matters most.

Versions

Earlier releases are kept as tags of this repo: v1 (Mizan 1.0) and v1.5 (Mizan 1.5).

License

Apache-2.0, the same as the Qwen3.5 base model.