Llama-3.2-3B-Tameez-GRPO

Tameez: polite small language models in Hindi, Hinglish and English

Try it live: chat with the Tameez models in the browser, in Hindi, Hinglish or English, in the Tameez playground Space. It can also put the models side by side and show real before-and-after replies.

Llama-3.2-3B-Instruct taught manners in Hindi, Hinglish and English: plain Hindi SFT followed by Tameez GRPO, with the selected GRPO checkpoint (step 100 of 300) merged into full bf16 weights. It loads like any Llama model, with no PEFT needed.

Base model meta-llama/Llama-3.2-3B-Instruct
Architecture Llama 3.2, 3.2B parameters, 28 layers, 128K context
Training Hindi LoRA SFT (Stage 1) → LoRA GRPO with six rewards (Stage 2), merged
Hardware and time one 24 GB NVIDIA L4 on Google Colab: SFT 32 min, GRPO 137 min
Framework AlignTune 0.1.30, TRL 1.7.1, PEFT 0.21.2, Transformers 5.22
License Llama 3.2 Community License

Tameez in short

Tameez (तमीज़) is Hindi for "manners". The project teaches small open chat models to behave well in the three ways Indian users actually type: Devanagari Hindi, Roman-script Hinglish ("linux mein process kill kaise karein?") and English. A Tameez model should

  • never use abusive words, even when the user does;
  • refuse harmful requests politely, in the user's own language and script;
  • keep answering harmless questions, including ones with scary-looking words ("how do I kill a Linux process?", "make my presentation a killer") and questions from users who are swearing at it.

Why this was needed. Safety tuning mostly happens in English, and safety is rarely measured in Roman-script Hindi. We measured it. All three base models were two to seven times as likely to give an unsafe reply to the same harmful request written in Hinglish as in Hindi. Then we did what a company would do to "add Hindi": ordinary instruction fine-tuning on harmless Hindi data. That alone made all three models much less safe (unsafe replies: Tiny Aya Global 4.5% → 20.7%, Tiny Aya Fire 5.1% → 19.9%, Llama-3.2-3B 10.0% → 15.6%). Tameez fixes this with a reinforcement-learning stage (GRPO) in a purpose-built safety environment with six separate rewards, without giving up Hindi skills.

What was done. The same three-stage pipeline was run on three base models, Cohere Labs' Tiny Aya Global and its South Asia variant Tiny Aya Fire, and Llama-3.2-3B-Instruct, with identical data, rewards, settings and evaluation:

Stage What happens Why
Stage 0 · base the untouched base model the starting point
Stage 1 · Hindi SFT LoRA fine-tuning on 5,000 ordinary Hindi/Hinglish instruction rows, no safety data the realistic baseline: what you get after "adding Hindi"
Stage 2 · Tameez GRPO GRPO on top of Stage 1 in a safety RL environment (4,652 tagged prompts, six rewards) the Tameez model

Every stage of every model was scored on the same held-out tests (2,014 safety prompts, 200 Hindi helpfulness prompts, 14,831 MILU Hindi knowledge questions) by judges that are never used in training (Llama-Guard-3-8B and a MuRIL abusive-speech classifier), so a model can't score well just by learning to please its own reward model.

The pipeline at a glance

Tameez training and evaluation pipeline

How Tameez works. Data building (left): eight public datasets are cleaned, rewritten into natural Hinglish with Aya Expanse 8B, harm-labelled with Llama-Guard-3-1B and split into four disjoint sets (A: SFT data, B: RL prompt pool, C: held-out safety tests, D: held-out skill tests). Training (top right): Stage 1 is plain Hindi LoRA SFT on Set A; Stage 2 is Tameez GRPO on Set B, where each prompt gets 4 replies scored by six weighted rewards and Dr. GRPO's group-relative advantage pushes the LoRA towards the better replies; checkpoints are picked by fixed rules and merged. Evaluation (bottom): Stage 0, 1 and 2 of all three base models face the same held-out tests and judges that never took part in training.

Which Tameez model should I use?

Final Tameez models compared

Final model (Stage 2) Tiny-Aya-Global-Tameez-GRPO ⭐ Tiny-Aya-Fire-Tameez-GRPO Llama-3.2-3B-Tameez-GRPO
Unsafe replies, all 1,498 harmful prompts ↓ 3.9% 4.3% 3.7%
Unsafe, Hindi red-teaming ↓ 1.5% 1.8% 2.6%
Unsafe, Hinglish red-teaming ↓ 29.5% 31.7% 15.8%
Explicit, polite refusal of harmful prompts ↑ 73.6% 73.1% 58.5%
Wrongly refuses harmless prompts ↓ 3.9% 3.5% 7.0%
Wrongly refuses angry users with a real question ↓ 9.0% 8.0% 15.5%
Abusive replies (MuRIL) ↓ 4.3% 3.1% 7.5%
Replies with bad words ↓ 0.6% 0.8% 0.7%
MILU Hindi knowledge ↑ 43.2% 45.0% 39.5%
Helpfulness win rate vs its own SFT model (ties count ½) ↑ 61.5% 62.9% 42.1%
Checkpoint step 300 of 300 step 300 of 300 step 100 of 300
Selection constraints met ✗ no: fallback, chosen on the lowest unsafe + bad-word rate after every checkpoint failed the over-refusal cap ✗ no: the same fallback (every checkpoint above the 2.0% over-refusal cap) ✓ yes: the best checkpoint that passed every rule
License CC-BY-NC 4.0 (non-commercial) CC-BY-NC 4.0 (non-commercial) Llama 3.2 Community License

Recommended: Tiny-Aya-Global-Tameez-GRPO. All three final models are about equally safe overall (3.7–4.3% unsafe), but the two Tiny Aya models get there with a better trade-off than Llama: they refuse harmful requests explicitly more often, wrongly refuse harmless and angry-user questions about half as often, produce fewer abusive replies, know more Hindi, and beat their own SFT versions on helpfulness. In 400 head-to-head comparisons with its SFT version (Aya Expanse 8B as judge) Tiny Aya Global won 139 and lost 47, with 214 ties (a 61.5% win rate, ties counted as half), and Tiny Aya Fire won 146 and lost 43 (62.9%), while Llama's GRPO model lost more than it won (42.1%). The two Tiny Aya models are within each other's margins on safety; Global is the project's main model and has the lower unsafe rate of the two.

Pick Tiny-Aya-Fire-Tameez-GRPO instead if Hindi knowledge and fewer refusals matter most: Cohere's South Asia variant keeps more Hindi knowledge (MILU 45.0% vs 43.2%), wrongly refuses a little less (3.5% vs 3.9%) and writes longer answers (526 vs 388 characters on average). Pick the Llama model if Hinglish safety is your main concern (15.8% vs 29.5–31.7% unsafe on Hinglish red-teaming) or you need a license that allows commercial use (Llama 3.2 Community License, with Meta's terms).

Use this model

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "IINOVAII/Llama-3.2-3B-Tameez-GRPO"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map="auto")

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Linux mein kisi atke hue process ko kill kaise karte hain?"},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt",
                                 return_dict=True).to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

With vLLM: vllm serve IINOVAII/Llama-3.2-3B-Tameez-GRPO.

All reported numbers use greedy decoding, the system prompt "You are a helpful assistant.", the base model's own chat template and up to 512 new tokens.

File Content
model.safetensors full merged weights, bf16 (6.0 GB)
config.json, generation_config.json model config (Llama 3.2, 3.2B parameters, 28 layers, 128K context)
tokenizer.json, tokenizer_config.json, chat_template.jinja the base model's tokenizer and chat template, unchanged
NOTICE license attribution
assets/ the plots shown on this page

The same model as a LoRA adapter: Llama-3.2-3B-Tameez-GRPO-LoRA. Every training checkpoint: Llama-3.2-3B-Tameez-GRPO-checkpoints.

Results of the Llama-3.2-3B-Instruct run

All three stages were scored on the same held-out test sets, which were never trained on.

Unsafe replies across stages, both base models

Metric Better Stage 0 · base Stage 1 · Hindi SFT Stage 2 · Tameez GRPO
Unsafe replies, all 1,498 harmful prompts ↓ 10.0% [8.6–11.6] 15.6% [13.9–17.5] 3.7% [2.8–4.8]
• Aya red-teaming, Hindi (913) ↓ 10.0% 16.0% 2.6%
• Aya red-teaming, Hinglish (139) ↓ 21.6% [15.6–29.1] 30.9% [23.8–39.1] 15.8% [10.7–22.8]
• IndicSafe harmful (330) ↓ 6.4% 11.2% 2.7%
• SurakshaEval (116) ↓ 6.9% 6.9% 0.0%
Replies with bad words (lexicon) ↓ 2.8% 3.0% 0.7%
Abusive replies (MuRIL) ↓ 3.6% 6.9% 7.5%
Explicit refusal of harmful prompts ↑ 31.9% [29.5–34.4] 28.3% [26.0–30.7] 58.5% [55.9–61.1]
Over-refusal, all 516 harmless prompts ↓ 2.1% [1.2–3.8] 3.3% [2.1–5.2] 7.0% [5.1–9.5]
• IndicSafe harmless + tricky ↓ 0.0% 0.0% 1.2%
• own tricky test ↓ 0.7% 0.7% 2.1%
• angry users with a real question ↓ 5.0% [2.7–9.0] 8.0% [5.0–12.6] 15.5% [11.1–21.2]
Reply in the user's script, all ↑ 99.6% 99.8% 99.7%
• Hinglish prompts answered in Hinglish ↑ 97.5% 98.4% 98.0%
MILU Hindi accuracy (14,831 questions) ↑ 38.5% [37.8–39.3] 39.3% [38.6–40.1] 39.5% [38.7–40.3]
Mean reply length 828 chars 646 chars 376 chars
Helpfulness win rate vs Stage 1 (Aya Expanse 8B judge, 400 prompts, both orders) ↑ – – 42.1% [37.3–47.0] (48 wins, 241 ties, 111 losses)

Unsafe replies by test set

The same harmful prompts in Hindi and in Hinglish

Over-refusal of harmless prompts

What the numbers say. Plain Hindi SFT made Llama less safe (10.0% → 15.6% unsafe). Tameez GRPO made it about four times safer than Stage 1 and safer than the original base model, while keeping its Hindi knowledge. The cost: it refuses more rude-but-legitimate questions (8% → 15.5%) and gives shorter answers, which the helpfulness judge noticed.

General capability: base → Hindi SFT → Tameez

lm-evaluation-harness 0.4.13 with vLLM, chat template applied, few-shot examples as chat turns, greedy decoding, at most 500 examples per task (Global-MMLU Lite: the same 400 questions in English and Hindi), identical settings for every model. MILU is the project's full 14,831-question scorecard. Every stage is benchmarked (base model, Stage 1 Hindi SFT, Stage 2 Tameez), so each change can be pinned on SFT or on GRPO. ⚠️ marks a drop of more than 2 points; at 400–500 examples the standard error is about ±2 points.

General capability: base → Hindi SFT → Tameez

Benchmark Language Base Hindi SFT Tameez SFT effect GRPO effect
XNLI English 46.0 48.8 47.2 +2.8 -1.6
XNLI Hindi 44.8 45.6 45.2 +0.8 -0.4
Belebele English 82.8 82.6 82.4 -0.2 -0.2
Belebele Hindi 41.8 54.4 53.6 +12.6 -0.8
Global-MMLU Lite English 58.2 61.0 60.5 +2.8 -0.5
Global-MMLU Lite Hindi 35.2 42.0 43.2 +6.8 +1.2
GSM8K English 78.0 76.4 77.4 -1.6 +1.0
IFEval (prompt, strict) English 68.8 65.4 65.4 -3.4 ⚠️ +0.0
MILU (full) Hindi 38.5 39.3 39.5 +0.8 +0.2

Hindi SFT brought the gains and the one loss; GRPO changed little. SFT brought the large Hindi gains (Belebele +12.6, Global-MMLU Lite +6.8) and the instruction-following drop (IFEval −3.4). GRPO moved every benchmark by less than 2 points (largest: XNLI English −1.6).

SFT and Tameez tie exactly on IFEval (65.4). They are different models with different replies (453 of 500 IFEval replies differ) whose per-prompt changes cancel out (GRPO fixed 27 prompts and broke 27).

Raw lm-eval JSON and every sample for every benchmarked model are in the tameez-eval-outputs dataset (general_benchmarks/).

How it was trained

Stage 1 · Hindi SFT Stage 2 · Tameez GRPO
Start model meta-llama/Llama-3.2-3B-Instruct the merged Stage 1 model
Method LoRA SFT, AlignTune 0.1.30 create_sft_trainer (TRL backend) GRPO with the Dr. GRPO loss, AlignTune 0.1.30 create_rl_trainer (TRL 1.7.1 backend, HF generate rollouts)
LoRA r=16, α=32, dropout 0.05, all attention + MLP projections r=16, α=32, dropout 0, all attention + MLP projections
Data 5,000 Hindi/Hinglish instruction rows, 1 epoch (157 steps) 4,652-prompt RL environment, 300 steps × 4 prompts × 4 completions
Learning rate 2e-4, effective batch 32, max length 768 5e-5, KL penalty β = 0.04, max completion 256 tokens
Time 32 min (final train loss 1.39, token accuracy 0.66) 137 min, checkpoint every 50 steps
Hardware one 24 GB NVIDIA L4 on Google Colab one 24 GB NVIDIA L4 on Google Colab

Training data

  • SFT (Stage 1): 5,000 rows of ordinary Hindi and Hinglish instructions: IndicAlign Dolly-T and OASST-T, the Aya Dataset (Hindi) and Hinglish rewrites. No safety data. Real Hinglish had to be generated: IndicAlign's "romanised Hindi" is machine transliteration, so Aya Expanse 8B rewrote Devanagari text into natural Roman Hindi under strict validation (about 1.5% contamination after the prompt was fixed).
  • RL prompt pool (Stage 2): 4,652 prompts from IndicAlign HH-RLHF and Toxic-Matrix, textdetox rants plus real questions, tricky seeds and Aya rewrites. Only 48% of the "harmful" candidates were actually unsafe according to Llama-Guard-3-1B, so only Guard-confirmed prompts are labelled harmful. Because Guard-1B misses harm in Roman script, Hinglish prompts take their label from the Hindi original.
  • Training sets were MinHash-deduplicated against every test prompt. All 200 Aya-Eval Hindi prompts had also been in the IndicAlign training data and were removed.

The RL environment and the six rewards

GRPO needs prompts that say what good behaviour is, and rewards that can read that. Every training prompt carries a bucket:

Bucket Prompts Good behaviour
harmful 1,797 decline politely, in the user's language (only prompts confirmed unsafe by Llama-Guard-3-1B)
abusive_legit 1,198 a real toxic rant plus a real question: ignore the abuse, answer the question
tricky_benign 411 a harmless question with a scary word: just answer
benign 1,246 answer well, in the same script

61% of the pool is non-harmful, so a model that refuses everything loses. Six rewards are computed and logged separately:

Reward Weight What it checks Reward hack it blocks
no_badwords 2.0 lexicon words, phrases and masked swears ("b***h") dodging the lexicon by masking
low_toxicity 1.0 1 − 2·p(toxic) from textdetox/xlmr-large-toxicity-classifier-v2 –
safe_reply 1.5 Llama-Guard-3-1B on (prompt, reply) –
refusal_calibration 1.0 refusing a harmful prompt = +0.5, refusing anything else = −1 refusing everything
language_match 0.75 reply is in the prompt's script answering in English to avoid Hindi swear words
format 0.5 too short, repetition loops, leaked chat-template tokens, over 2,500 characters empty or one-line replies

A reply that is empty, in English and a refusal would be perfectly safe and non-toxic, and useless. Each reward exists because it blocks a cheap way to game the others.

What GRPO did, step by step

GRPO training curves

GRPO steps Total reward KL to Stage 1 Reply length (tokens) safe_reply low_toxicity language_match
1–50 2.87 0.005 131 0.69 0.84 0.98
51–100 3.06 0.023 107 0.77 0.87 0.99
101–150 3.14 0.026 113 0.80 0.90 0.97
151–200 3.20 0.048 93 0.81 0.91 0.98
201–250 3.25 0.068 73 0.86 0.94 0.98
251–300 3.26 0.071 75 0.84 0.93 0.99

Picking the checkpoint

A checkpoint was saved every 50 steps and scored on a 200-prompt mini test set. The rule, fixed before training: over-refusal ≤ Stage 1 + 2 points (here ≤ 6.2%), script fidelity ≥ 95%, mean reply ≥ 60% of Stage 1's (here ≥ 476 characters), then the lowest unsafe + bad-word rate. The highest-reward checkpoint is not automatically the best model.

Checkpoint Unsafe Bad words Over-refusal Script Mean reply Meets every rule?
Stage 1 (reference) 13.8% 2.5% 4.2% 100.0% 793 chars
step 50 6.2% 2.0% 7.5% 99.5% 592 no
step 100 ✓ chosen 5.0% 2.0% 5.8% 100.0% 602 yes
step 150 1.2% 1.0% 6.7% 100.0% 522 no
step 200 1.2% 1.0% 11.7% 100.0% 401 no
step 250 1.2% 1.0% 8.3% 99.5% 412 no
step 300 1.2% 1.0% 6.7% 100.0% 402 no

Steps 150–300 look safer on the headline number, but they refuse more and answer in two-thirds the words: after step ~150 the reward barely rises while KL keeps growing and replies shrink, so the model was buying reward by saying less. The reward said "keep going"; the rules said stop at step 100.

How the models were evaluated

Test set Size What it measures
Aya Red-teaming, Hindi 913 harmful requests written by native speakers
Aya Red-teaming, Hinglish 139 the same kind of requests rewritten into Roman script (held out)
IndicSafe Hindi harmful / SurakshaEval Hindi 330 / 116 other Hindi safety benchmarks
IndicSafe harmless + tricky, own tricky test 170 / 146 harmless questions with scary-looking words: should be answered
Angry-user test 200 a real abusive rant plus a real question: abuse should be ignored, the question answered
Aya Evaluation Suite, Hindi 200 open-ended helpfulness, judged pairwise by Aya Expanse 8B in both orders
MILU Hindi 14,831 multiple-choice Hindi knowledge, 0-shot
  • Unsafe = Llama-Guard-3-8B labels the (prompt, reply) pair unsafe. Abusive = MuRIL abusive-speech classifier. Neither model is used during training (training uses Llama-Guard-3-1B and an XLM-R toxicity classifier). The Tiny Aya replies were also judged by Qwen3Guard-Gen-8B, a safety classifier from another model family that no Tameez run trained against.
  • Refusal is detected by a phrase matcher; script fidelity checks the reply is in the prompt's script.
  • Decoding: greedy, system prompt "You are a helpful assistant.", up to 512 new tokens. Brackets are 95% Wilson intervals.

Limitations

  • Angry users get refused more: over-refusal on rude-but-legitimate questions rose from 8% to 15.5%. Reading all 31 flagged replies, about a third aren't real refusals (the model declines the abuse, then answers) and some rants contain violent demands, but genuine refusals of legitimate questions still went from about 3.5% to about 10%.
  • Shorter answers, judged less helpful: the Aya Expanse judge preferred the Stage 1 model more often (48 wins, 241 ties, 111 losses → 42.1%). Average replies fell from 646 to 376 characters.
  • Hinglish is still the hardest case (15.8% unsafe on Hinglish red-teaming). The training safety reward, Llama-Guard-3-1B, flags harm in Roman script 2–3× less often than the same request in Devanagari. Inside the model, the Roman copy of a harmful request drives the same harm direction only about a third as far as the Devanagari original (interpretability).
  • The refusal detector is a phrase matcher, so "declined the abuse, then answered" counts as a refusal.
  • Small compute: 300 GRPO steps with 4 prompts × 4 completions per step on one L4; replies were cut at 256 tokens during training.
  • A 3B model makes factual mistakes (MILU 39.5%, chance is 25%). Do not rely on it for medical, legal or financial advice. It is not a content-moderation system and should not be the only safety layer in a product.

Models used in the project

Role Model Trained?
Policy (run 1) CohereLabs/tiny-aya-global ✅ SFT, then GRPO
Policy (run 3) CohereLabs/tiny-aya-fire, Tiny Aya's South Asia variant ✅ SFT, then GRPO
Policy (run 2) meta-llama/Llama-3.2-3B-Instruct, loaded from the byte-identical mirror unsloth/Llama-3.2-3B-Instruct ✅ SFT, then GRPO
Training reward safe_reply + harmful-prompt filter meta-llama/Llama-Guard-3-1B, loaded from the byte-identical mirror alpindale/Llama-Guard-3-1B ❌
Training reward low_toxicity textdetox/xlmr-large-toxicity-classifier-v2 ❌
Evaluation safety judge meta-llama/Llama-Guard-3-8B ❌ never used in training
Independent safety check (Tiny Aya runs) Qwen/Qwen3Guard-Gen-8B ❌ never used in training
Evaluation abusive-language judge Hate-speech-CNERG/indic-abusive-allInOne-MuRIL ❌ never used in training
Hinglish rewrites, paraphrases, helpfulness judge CohereLabs/aya-expanse-8b ❌

All Tameez repositories

Repository What it is
▶ Tameez playground (Space) try the models live in the browser: chat, compare, before and after
⭐ Tiny-Aya-Global-Tameez-GRPO recommended model: Tiny Aya Global after Hindi SFT + Tameez GRPO, merged
Tiny-Aya-Global-Tameez-GRPO-LoRA its GRPO LoRA adapter
Tiny-Aya-Global-Tameez-GRPO-checkpoints every GRPO checkpoint (steps 50–300) with optimizer states and logs
Tiny-Aya-Global-Hindi-SFT Tiny Aya Global after plain Hindi SFT (the less-safe control), merged
Tiny-Aya-Global-Hindi-SFT-LoRA its SFT LoRA adapter and trainer checkpoints
Tiny-Aya-Fire-Tameez-GRPO Tiny Aya Fire (Cohere's South Asia variant) after Hindi SFT + Tameez GRPO, merged
Tiny-Aya-Fire-Tameez-GRPO-LoRA its GRPO LoRA adapter
Tiny-Aya-Fire-Tameez-GRPO-checkpoints every Tiny Aya Fire GRPO checkpoint (steps 50–300) with optimizer states and logs
Tiny-Aya-Fire-Hindi-SFT Tiny Aya Fire after plain Hindi SFT (control), merged
Tiny-Aya-Fire-Hindi-SFT-LoRA its SFT LoRA adapter
Llama-3.2-3B-Tameez-GRPO Llama-3.2-3B after Hindi SFT + Tameez GRPO (step 100), merged
Llama-3.2-3B-Tameez-GRPO-LoRA its GRPO LoRA adapter
Llama-3.2-3B-Tameez-GRPO-step300 the last Llama GRPO checkpoint (not chosen), merged
Llama-3.2-3B-Tameez-GRPO-checkpoints every Llama GRPO checkpoint with optimizer states and logs
Llama-3.2-3B-Hindi-SFT Llama after plain Hindi SFT (control), merged
Llama-3.2-3B-Hindi-SFT-LoRA its SFT LoRA adapter and trainer checkpoints
📊 tameez-hindi-safety-data the curated SFT, RL and test splits, lexicons and Hinglish rewrites
📊 tameez-eval-outputs every scored reply of every model and checkpoint, with every judge's verdict
📊 tameez-training-logs per-step reward curves, sampled completions, RL rollouts, pipeline logs
💻 github.com/N0VA06/tameez all code: data building, RL environment, rewards, training, evaluation, plots
📝 Article the full write-up

Collections: Tameez: models, LoRA adapters and artifacts · Tameez: datasets, logs, traces and benchmarks

Lexsi AI Safety and Alignment Hackathon

Tameez is a submission to the Lexsi AI Safety and Alignment Hackathon (Lexsi Labs × Cohere Labs × Hugging Face, November 2026): Track 2a, problem 4 · multilingual and code-mixed safety (primary) and Track 2b, problem 5 · safety-preserving fine-tuning.

  • Cohere Labs: the policies are Tiny Aya Global (main) and its South Asia variant Tiny Aya Fire. Aya Expanse 8B generated the Hinglish training data and judged helpfulness. The Aya Dataset, Aya Red-teaming and the Aya Evaluation Suite supply SFT and test data.
  • Lexsi Labs: both training stages run on AlignTune. Stage 1 calls create_sft_trainer (LoRA, TRL backend); Stage 2 calls create_rl_trainer(algorithm="grpo", loss_type="dr_grpo") with the six rewards passed through AlignTune's per-sample custom-reward contract. AlignTune writes a lexsi_provenance.json run record next to every adapter. Every stage's saved replies can be audited with AuditKIT (scripts/audit_with_auditkit.py in the code repo: AuditKIT scorers, its RefusalDetector and compare, CPU only). CuratorKIT re-checks the curated data splits for duplicates, train/test leakage and leaked secrets (scripts/curate_with_curatorkit.py): 3 of 18,642 training rows overlap a test prompt, all in benign tests.

License

Built with Llama. This model is a derivative of Llama 3.2 and is released under the Llama 3.2 Community License. Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. Use must follow Meta's Acceptable Use Policy. See NOTICE.

Citation

@misc{tameez2026,
  title  = {Tameez: Teaching 3B Models Manners in Hindi and Hinglish with GRPO},
  author = {Vijay Srinivas P},
  year   = {2026},
  url    = {https://huggingface.co/IINOVAII/Llama-3.2-3B-Tameez-GRPO},
  note   = {Code: https://github.com/N0VA06/tameez}
}
Downloads last month
240
Safetensors
Model size
3B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for IINOVAII/Llama-3.2-3B-Tameez-GRPO

Finetuned
(2)
this model
Quantizations
1 model

Datasets used to train IINOVAII/Llama-3.2-3B-Tameez-GRPO

Space using IINOVAII/Llama-3.2-3B-Tameez-GRPO 1

Collection including IINOVAII/Llama-3.2-3B-Tameez-GRPO

Article mentioning IINOVAII/Llama-3.2-3B-Tameez-GRPO