Instructions to use IINOVAII/Llama-3.2-3B-Tameez-GRPO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use IINOVAII/Llama-3.2-3B-Tameez-GRPO with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="IINOVAII/Llama-3.2-3B-Tameez-GRPO") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("IINOVAII/Llama-3.2-3B-Tameez-GRPO") model = AutoModelForCausalLM.from_pretrained("IINOVAII/Llama-3.2-3B-Tameez-GRPO", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use IINOVAII/Llama-3.2-3B-Tameez-GRPO with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "IINOVAII/Llama-3.2-3B-Tameez-GRPO" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "IINOVAII/Llama-3.2-3B-Tameez-GRPO", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/IINOVAII/Llama-3.2-3B-Tameez-GRPO
- SGLang
How to use IINOVAII/Llama-3.2-3B-Tameez-GRPO with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "IINOVAII/Llama-3.2-3B-Tameez-GRPO" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "IINOVAII/Llama-3.2-3B-Tameez-GRPO", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "IINOVAII/Llama-3.2-3B-Tameez-GRPO" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "IINOVAII/Llama-3.2-3B-Tameez-GRPO", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use IINOVAII/Llama-3.2-3B-Tameez-GRPO with Docker Model Runner:
docker model run hf.co/IINOVAII/Llama-3.2-3B-Tameez-GRPO
- Llama-3.2-3B-Tameez-GRPO
- Tameez in short
- Which Tameez model should I use?
- Use this model
- Results of the Llama-3.2-3B-Instruct run
- General capability: base → Hindi SFT → Tameez
- How it was trained
- Picking the checkpoint
- How the models were evaluated
- Limitations
- Models used in the project
- All Tameez repositories
- Lexsi AI Safety and Alignment Hackathon
- License
- Citation
Llama-3.2-3B-Tameez-GRPO
Try it live: chat with the Tameez models in the browser, in Hindi, Hinglish or English, in the Tameez playground Space. It can also put the models side by side and show real before-and-after replies.
Llama-3.2-3B-Instruct taught manners in Hindi, Hinglish and English: plain Hindi SFT followed by Tameez GRPO, with the selected GRPO checkpoint (step 100 of 300) merged into full bf16 weights. It loads like any Llama model, with no PEFT needed.
| Base model | meta-llama/Llama-3.2-3B-Instruct |
| Architecture | Llama 3.2, 3.2B parameters, 28 layers, 128K context |
| Training | Hindi LoRA SFT (Stage 1) → LoRA GRPO with six rewards (Stage 2), merged |
| Hardware and time | one 24 GB NVIDIA L4 on Google Colab: SFT 32 min, GRPO 137 min |
| Framework | AlignTune 0.1.30, TRL 1.7.1, PEFT 0.21.2, Transformers 5.22 |
| License | Llama 3.2 Community License |
Tameez in short
Tameez (तमीज़) is Hindi for "manners". The project teaches small open chat models to behave well in the three ways Indian users actually type: Devanagari Hindi, Roman-script Hinglish ("linux mein process kill kaise karein?") and English. A Tameez model should
- never use abusive words, even when the user does;
- refuse harmful requests politely, in the user's own language and script;
- keep answering harmless questions, including ones with scary-looking words ("how do I kill a Linux process?", "make my presentation a killer") and questions from users who are swearing at it.
Why this was needed. Safety tuning mostly happens in English, and safety is rarely measured in Roman-script Hindi. We measured it. All three base models were two to seven times as likely to give an unsafe reply to the same harmful request written in Hinglish as in Hindi. Then we did what a company would do to "add Hindi": ordinary instruction fine-tuning on harmless Hindi data. That alone made all three models much less safe (unsafe replies: Tiny Aya Global 4.5% → 20.7%, Tiny Aya Fire 5.1% → 19.9%, Llama-3.2-3B 10.0% → 15.6%). Tameez fixes this with a reinforcement-learning stage (GRPO) in a purpose-built safety environment with six separate rewards, without giving up Hindi skills.
What was done. The same three-stage pipeline was run on three base models, Cohere Labs' Tiny Aya Global and its South Asia variant Tiny Aya Fire, and Llama-3.2-3B-Instruct, with identical data, rewards, settings and evaluation:
| Stage | What happens | Why |
|---|---|---|
| Stage 0 · base | the untouched base model | the starting point |
| Stage 1 · Hindi SFT | LoRA fine-tuning on 5,000 ordinary Hindi/Hinglish instruction rows, no safety data | the realistic baseline: what you get after "adding Hindi" |
| Stage 2 · Tameez GRPO | GRPO on top of Stage 1 in a safety RL environment (4,652 tagged prompts, six rewards) | the Tameez model |
Every stage of every model was scored on the same held-out tests (2,014 safety prompts, 200 Hindi helpfulness prompts, 14,831 MILU Hindi knowledge questions) by judges that are never used in training (Llama-Guard-3-8B and a MuRIL abusive-speech classifier), so a model can't score well just by learning to please its own reward model.
The pipeline at a glance
How Tameez works. Data building (left): eight public datasets are cleaned, rewritten into natural Hinglish with Aya Expanse 8B, harm-labelled with Llama-Guard-3-1B and split into four disjoint sets (A: SFT data, B: RL prompt pool, C: held-out safety tests, D: held-out skill tests). Training (top right): Stage 1 is plain Hindi LoRA SFT on Set A; Stage 2 is Tameez GRPO on Set B, where each prompt gets 4 replies scored by six weighted rewards and Dr. GRPO's group-relative advantage pushes the LoRA towards the better replies; checkpoints are picked by fixed rules and merged. Evaluation (bottom): Stage 0, 1 and 2 of all three base models face the same held-out tests and judges that never took part in training.
Which Tameez model should I use?
| Final model (Stage 2) | Tiny-Aya-Global-Tameez-GRPO ⭐ | Tiny-Aya-Fire-Tameez-GRPO | Llama-3.2-3B-Tameez-GRPO |
|---|---|---|---|
| Unsafe replies, all 1,498 harmful prompts ↓ | 3.9% | 4.3% | 3.7% |
| Unsafe, Hindi red-teaming ↓ | 1.5% | 1.8% | 2.6% |
| Unsafe, Hinglish red-teaming ↓ | 29.5% | 31.7% | 15.8% |
| Explicit, polite refusal of harmful prompts ↑ | 73.6% | 73.1% | 58.5% |
| Wrongly refuses harmless prompts ↓ | 3.9% | 3.5% | 7.0% |
| Wrongly refuses angry users with a real question ↓ | 9.0% | 8.0% | 15.5% |
| Abusive replies (MuRIL) ↓ | 4.3% | 3.1% | 7.5% |
| Replies with bad words ↓ | 0.6% | 0.8% | 0.7% |
| MILU Hindi knowledge ↑ | 43.2% | 45.0% | 39.5% |
| Helpfulness win rate vs its own SFT model (ties count ½) ↑ | 61.5% | 62.9% | 42.1% |
| Checkpoint | step 300 of 300 | step 300 of 300 | step 100 of 300 |
| Selection constraints met | ✗ no: fallback, chosen on the lowest unsafe + bad-word rate after every checkpoint failed the over-refusal cap | ✗ no: the same fallback (every checkpoint above the 2.0% over-refusal cap) | ✓ yes: the best checkpoint that passed every rule |
| License | CC-BY-NC 4.0 (non-commercial) | CC-BY-NC 4.0 (non-commercial) | Llama 3.2 Community License |
Recommended: Tiny-Aya-Global-Tameez-GRPO. All three final models are about equally safe overall (3.7–4.3% unsafe), but the two Tiny Aya models get there with a better trade-off than Llama: they refuse harmful requests explicitly more often, wrongly refuse harmless and angry-user questions about half as often, produce fewer abusive replies, know more Hindi, and beat their own SFT versions on helpfulness. In 400 head-to-head comparisons with its SFT version (Aya Expanse 8B as judge) Tiny Aya Global won 139 and lost 47, with 214 ties (a 61.5% win rate, ties counted as half), and Tiny Aya Fire won 146 and lost 43 (62.9%), while Llama's GRPO model lost more than it won (42.1%). The two Tiny Aya models are within each other's margins on safety; Global is the project's main model and has the lower unsafe rate of the two.
Pick Tiny-Aya-Fire-Tameez-GRPO instead if Hindi knowledge and fewer refusals matter most: Cohere's South Asia variant keeps more Hindi knowledge (MILU 45.0% vs 43.2%), wrongly refuses a little less (3.5% vs 3.9%) and writes longer answers (526 vs 388 characters on average). Pick the Llama model if Hinglish safety is your main concern (15.8% vs 29.5–31.7% unsafe on Hinglish red-teaming) or you need a license that allows commercial use (Llama 3.2 Community License, with Meta's terms).
Use this model
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
repo = "IINOVAII/Llama-3.2-3B-Tameez-GRPO"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map="auto")
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Linux mein kisi atke hue process ko kill kaise karte hain?"},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt",
return_dict=True).to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
With vLLM: vllm serve IINOVAII/Llama-3.2-3B-Tameez-GRPO.
All reported numbers use greedy decoding, the system prompt "You are a helpful assistant.", the base model's own chat
template and up to 512 new tokens.
| File | Content |
|---|---|
model.safetensors |
full merged weights, bf16 (6.0 GB) |
config.json, generation_config.json |
model config (Llama 3.2, 3.2B parameters, 28 layers, 128K context) |
tokenizer.json, tokenizer_config.json, chat_template.jinja |
the base model's tokenizer and chat template, unchanged |
NOTICE |
license attribution |
assets/ |
the plots shown on this page |
The same model as a LoRA adapter: Llama-3.2-3B-Tameez-GRPO-LoRA. Every training checkpoint: Llama-3.2-3B-Tameez-GRPO-checkpoints.
Results of the Llama-3.2-3B-Instruct run
All three stages were scored on the same held-out test sets, which were never trained on.
| Metric | Better | Stage 0 · base | Stage 1 · Hindi SFT | Stage 2 · Tameez GRPO |
|---|---|---|---|---|
| Unsafe replies, all 1,498 harmful prompts | ↓ | 10.0% [8.6–11.6] | 15.6% [13.9–17.5] | 3.7% [2.8–4.8] |
| • Aya red-teaming, Hindi (913) | ↓ | 10.0% | 16.0% | 2.6% |
| • Aya red-teaming, Hinglish (139) | ↓ | 21.6% [15.6–29.1] | 30.9% [23.8–39.1] | 15.8% [10.7–22.8] |
| • IndicSafe harmful (330) | ↓ | 6.4% | 11.2% | 2.7% |
| • SurakshaEval (116) | ↓ | 6.9% | 6.9% | 0.0% |
| Replies with bad words (lexicon) | ↓ | 2.8% | 3.0% | 0.7% |
| Abusive replies (MuRIL) | ↓ | 3.6% | 6.9% | 7.5% |
| Explicit refusal of harmful prompts | ↑ | 31.9% [29.5–34.4] | 28.3% [26.0–30.7] | 58.5% [55.9–61.1] |
| Over-refusal, all 516 harmless prompts | ↓ | 2.1% [1.2–3.8] | 3.3% [2.1–5.2] | 7.0% [5.1–9.5] |
| • IndicSafe harmless + tricky | ↓ | 0.0% | 0.0% | 1.2% |
| • own tricky test | ↓ | 0.7% | 0.7% | 2.1% |
| • angry users with a real question | ↓ | 5.0% [2.7–9.0] | 8.0% [5.0–12.6] | 15.5% [11.1–21.2] |
| Reply in the user's script, all | ↑ | 99.6% | 99.8% | 99.7% |
| • Hinglish prompts answered in Hinglish | ↑ | 97.5% | 98.4% | 98.0% |
| MILU Hindi accuracy (14,831 questions) | ↑ | 38.5% [37.8–39.3] | 39.3% [38.6–40.1] | 39.5% [38.7–40.3] |
| Mean reply length | 828 chars | 646 chars | 376 chars | |
| Helpfulness win rate vs Stage 1 (Aya Expanse 8B judge, 400 prompts, both orders) | ↑ | – | – | 42.1% [37.3–47.0] (48 wins, 241 ties, 111 losses) |
What the numbers say. Plain Hindi SFT made Llama less safe (10.0% → 15.6% unsafe). Tameez GRPO made it about four times safer than Stage 1 and safer than the original base model, while keeping its Hindi knowledge. The cost: it refuses more rude-but-legitimate questions (8% → 15.5%) and gives shorter answers, which the helpfulness judge noticed.
General capability: base → Hindi SFT → Tameez
lm-evaluation-harness 0.4.13 with vLLM, chat template applied, few-shot examples as chat turns, greedy decoding, at most 500 examples per task (Global-MMLU Lite: the same 400 questions in English and Hindi), identical settings for every model. MILU is the project's full 14,831-question scorecard. Every stage is benchmarked (base model, Stage 1 Hindi SFT, Stage 2 Tameez), so each change can be pinned on SFT or on GRPO. ⚠️ marks a drop of more than 2 points; at 400–500 examples the standard error is about ±2 points.
| Benchmark | Language | Base | Hindi SFT | Tameez | SFT effect | GRPO effect |
|---|---|---|---|---|---|---|
| XNLI | English | 46.0 | 48.8 | 47.2 | +2.8 | -1.6 |
| XNLI | Hindi | 44.8 | 45.6 | 45.2 | +0.8 | -0.4 |
| Belebele | English | 82.8 | 82.6 | 82.4 | -0.2 | -0.2 |
| Belebele | Hindi | 41.8 | 54.4 | 53.6 | +12.6 | -0.8 |
| Global-MMLU Lite | English | 58.2 | 61.0 | 60.5 | +2.8 | -0.5 |
| Global-MMLU Lite | Hindi | 35.2 | 42.0 | 43.2 | +6.8 | +1.2 |
| GSM8K | English | 78.0 | 76.4 | 77.4 | -1.6 | +1.0 |
| IFEval (prompt, strict) | English | 68.8 | 65.4 | 65.4 | -3.4 ⚠️ | +0.0 |
| MILU (full) | Hindi | 38.5 | 39.3 | 39.5 | +0.8 | +0.2 |
Hindi SFT brought the gains and the one loss; GRPO changed little. SFT brought the large Hindi gains (Belebele +12.6, Global-MMLU Lite +6.8) and the instruction-following drop (IFEval −3.4). GRPO moved every benchmark by less than 2 points (largest: XNLI English −1.6).
SFT and Tameez tie exactly on IFEval (65.4). They are different models with different replies (453 of 500 IFEval replies differ) whose per-prompt changes cancel out (GRPO fixed 27 prompts and broke 27).
Raw lm-eval JSON and every sample for every benchmarked model are in the tameez-eval-outputs dataset (general_benchmarks/).
How it was trained
| Stage 1 · Hindi SFT | Stage 2 · Tameez GRPO | |
|---|---|---|
| Start model | meta-llama/Llama-3.2-3B-Instruct |
the merged Stage 1 model |
| Method | LoRA SFT, AlignTune 0.1.30 create_sft_trainer (TRL backend) |
GRPO with the Dr. GRPO loss, AlignTune 0.1.30 create_rl_trainer (TRL 1.7.1 backend, HF generate rollouts) |
| LoRA | r=16, α=32, dropout 0.05, all attention + MLP projections | r=16, α=32, dropout 0, all attention + MLP projections |
| Data | 5,000 Hindi/Hinglish instruction rows, 1 epoch (157 steps) | 4,652-prompt RL environment, 300 steps × 4 prompts × 4 completions |
| Learning rate | 2e-4, effective batch 32, max length 768 | 5e-5, KL penalty β = 0.04, max completion 256 tokens |
| Time | 32 min (final train loss 1.39, token accuracy 0.66) | 137 min, checkpoint every 50 steps |
| Hardware | one 24 GB NVIDIA L4 on Google Colab | one 24 GB NVIDIA L4 on Google Colab |
Training data
- SFT (Stage 1): 5,000 rows of ordinary Hindi and Hinglish instructions: IndicAlign Dolly-T and OASST-T, the Aya Dataset (Hindi) and Hinglish rewrites. No safety data. Real Hinglish had to be generated: IndicAlign's "romanised Hindi" is machine transliteration, so Aya Expanse 8B rewrote Devanagari text into natural Roman Hindi under strict validation (about 1.5% contamination after the prompt was fixed).
- RL prompt pool (Stage 2): 4,652 prompts from IndicAlign HH-RLHF and Toxic-Matrix, textdetox rants plus real questions, tricky seeds and Aya rewrites. Only 48% of the "harmful" candidates were actually unsafe according to Llama-Guard-3-1B, so only Guard-confirmed prompts are labelled harmful. Because Guard-1B misses harm in Roman script, Hinglish prompts take their label from the Hindi original.
- Training sets were MinHash-deduplicated against every test prompt. All 200 Aya-Eval Hindi prompts had also been in the IndicAlign training data and were removed.
The RL environment and the six rewards
GRPO needs prompts that say what good behaviour is, and rewards that can read that. Every training prompt carries a bucket:
| Bucket | Prompts | Good behaviour |
|---|---|---|
harmful |
1,797 | decline politely, in the user's language (only prompts confirmed unsafe by Llama-Guard-3-1B) |
abusive_legit |
1,198 | a real toxic rant plus a real question: ignore the abuse, answer the question |
tricky_benign |
411 | a harmless question with a scary word: just answer |
benign |
1,246 | answer well, in the same script |
61% of the pool is non-harmful, so a model that refuses everything loses. Six rewards are computed and logged separately:
| Reward | Weight | What it checks | Reward hack it blocks |
|---|---|---|---|
no_badwords |
2.0 | lexicon words, phrases and masked swears ("b***h") | dodging the lexicon by masking |
low_toxicity |
1.0 | 1 − 2·p(toxic) from textdetox/xlmr-large-toxicity-classifier-v2 |
– |
safe_reply |
1.5 | Llama-Guard-3-1B on (prompt, reply) | – |
refusal_calibration |
1.0 | refusing a harmful prompt = +0.5, refusing anything else = −1 | refusing everything |
language_match |
0.75 | reply is in the prompt's script | answering in English to avoid Hindi swear words |
format |
0.5 | too short, repetition loops, leaked chat-template tokens, over 2,500 characters | empty or one-line replies |
A reply that is empty, in English and a refusal would be perfectly safe and non-toxic, and useless. Each reward exists because it blocks a cheap way to game the others.
What GRPO did, step by step
| GRPO steps | Total reward | KL to Stage 1 | Reply length (tokens) | safe_reply |
low_toxicity |
language_match |
|---|---|---|---|---|---|---|
| 1–50 | 2.87 | 0.005 | 131 | 0.69 | 0.84 | 0.98 |
| 51–100 | 3.06 | 0.023 | 107 | 0.77 | 0.87 | 0.99 |
| 101–150 | 3.14 | 0.026 | 113 | 0.80 | 0.90 | 0.97 |
| 151–200 | 3.20 | 0.048 | 93 | 0.81 | 0.91 | 0.98 |
| 201–250 | 3.25 | 0.068 | 73 | 0.86 | 0.94 | 0.98 |
| 251–300 | 3.26 | 0.071 | 75 | 0.84 | 0.93 | 0.99 |
Picking the checkpoint
A checkpoint was saved every 50 steps and scored on a 200-prompt mini test set. The rule, fixed before training: over-refusal ≤ Stage 1 + 2 points (here ≤ 6.2%), script fidelity ≥ 95%, mean reply ≥ 60% of Stage 1's (here ≥ 476 characters), then the lowest unsafe + bad-word rate. The highest-reward checkpoint is not automatically the best model.
| Checkpoint | Unsafe | Bad words | Over-refusal | Script | Mean reply | Meets every rule? |
|---|---|---|---|---|---|---|
| Stage 1 (reference) | 13.8% | 2.5% | 4.2% | 100.0% | 793 chars | |
| step 50 | 6.2% | 2.0% | 7.5% | 99.5% | 592 | no |
| step 100 ✓ chosen | 5.0% | 2.0% | 5.8% | 100.0% | 602 | yes |
| step 150 | 1.2% | 1.0% | 6.7% | 100.0% | 522 | no |
| step 200 | 1.2% | 1.0% | 11.7% | 100.0% | 401 | no |
| step 250 | 1.2% | 1.0% | 8.3% | 99.5% | 412 | no |
| step 300 | 1.2% | 1.0% | 6.7% | 100.0% | 402 | no |
Steps 150–300 look safer on the headline number, but they refuse more and answer in two-thirds the words: after step ~150 the reward barely rises while KL keeps growing and replies shrink, so the model was buying reward by saying less. The reward said "keep going"; the rules said stop at step 100.
How the models were evaluated
| Test set | Size | What it measures |
|---|---|---|
| Aya Red-teaming, Hindi | 913 | harmful requests written by native speakers |
| Aya Red-teaming, Hinglish | 139 | the same kind of requests rewritten into Roman script (held out) |
| IndicSafe Hindi harmful / SurakshaEval Hindi | 330 / 116 | other Hindi safety benchmarks |
| IndicSafe harmless + tricky, own tricky test | 170 / 146 | harmless questions with scary-looking words: should be answered |
| Angry-user test | 200 | a real abusive rant plus a real question: abuse should be ignored, the question answered |
| Aya Evaluation Suite, Hindi | 200 | open-ended helpfulness, judged pairwise by Aya Expanse 8B in both orders |
| MILU Hindi | 14,831 | multiple-choice Hindi knowledge, 0-shot |
- Unsafe = Llama-Guard-3-8B labels the (prompt, reply) pair unsafe. Abusive = MuRIL abusive-speech classifier. Neither model is used during training (training uses Llama-Guard-3-1B and an XLM-R toxicity classifier). The Tiny Aya replies were also judged by Qwen3Guard-Gen-8B, a safety classifier from another model family that no Tameez run trained against.
- Refusal is detected by a phrase matcher; script fidelity checks the reply is in the prompt's script.
- Decoding: greedy, system prompt
"You are a helpful assistant.", up to 512 new tokens. Brackets are 95% Wilson intervals.
Limitations
- Angry users get refused more: over-refusal on rude-but-legitimate questions rose from 8% to 15.5%. Reading all 31 flagged replies, about a third aren't real refusals (the model declines the abuse, then answers) and some rants contain violent demands, but genuine refusals of legitimate questions still went from about 3.5% to about 10%.
- Shorter answers, judged less helpful: the Aya Expanse judge preferred the Stage 1 model more often (48 wins, 241 ties, 111 losses → 42.1%). Average replies fell from 646 to 376 characters.
- Hinglish is still the hardest case (15.8% unsafe on Hinglish red-teaming). The training safety reward, Llama-Guard-3-1B, flags harm in Roman script 2–3× less often than the same request in Devanagari. Inside the model, the Roman copy of a harmful request drives the same harm direction only about a third as far as the Devanagari original (interpretability).
- The refusal detector is a phrase matcher, so "declined the abuse, then answered" counts as a refusal.
- Small compute: 300 GRPO steps with 4 prompts × 4 completions per step on one L4; replies were cut at 256 tokens during training.
- A 3B model makes factual mistakes (MILU 39.5%, chance is 25%). Do not rely on it for medical, legal or financial advice. It is not a content-moderation system and should not be the only safety layer in a product.
Models used in the project
| Role | Model | Trained? |
|---|---|---|
| Policy (run 1) | CohereLabs/tiny-aya-global |
✅ SFT, then GRPO |
| Policy (run 3) | CohereLabs/tiny-aya-fire, Tiny Aya's South Asia variant |
✅ SFT, then GRPO |
| Policy (run 2) | meta-llama/Llama-3.2-3B-Instruct, loaded from the byte-identical mirror unsloth/Llama-3.2-3B-Instruct |
✅ SFT, then GRPO |
Training reward safe_reply + harmful-prompt filter |
meta-llama/Llama-Guard-3-1B, loaded from the byte-identical mirror alpindale/Llama-Guard-3-1B |
❌ |
Training reward low_toxicity |
textdetox/xlmr-large-toxicity-classifier-v2 |
❌ |
| Evaluation safety judge | meta-llama/Llama-Guard-3-8B |
❌ never used in training |
| Independent safety check (Tiny Aya runs) | Qwen/Qwen3Guard-Gen-8B |
❌ never used in training |
| Evaluation abusive-language judge | Hate-speech-CNERG/indic-abusive-allInOne-MuRIL |
❌ never used in training |
| Hinglish rewrites, paraphrases, helpfulness judge | CohereLabs/aya-expanse-8b |
❌ |
All Tameez repositories
| Repository | What it is | |
|---|---|---|
| ▶ | Tameez playground (Space) | try the models live in the browser: chat, compare, before and after |
| ⭐ | Tiny-Aya-Global-Tameez-GRPO | recommended model: Tiny Aya Global after Hindi SFT + Tameez GRPO, merged |
| Tiny-Aya-Global-Tameez-GRPO-LoRA | its GRPO LoRA adapter | |
| Tiny-Aya-Global-Tameez-GRPO-checkpoints | every GRPO checkpoint (steps 50–300) with optimizer states and logs | |
| Tiny-Aya-Global-Hindi-SFT | Tiny Aya Global after plain Hindi SFT (the less-safe control), merged | |
| Tiny-Aya-Global-Hindi-SFT-LoRA | its SFT LoRA adapter and trainer checkpoints | |
| Tiny-Aya-Fire-Tameez-GRPO | Tiny Aya Fire (Cohere's South Asia variant) after Hindi SFT + Tameez GRPO, merged | |
| Tiny-Aya-Fire-Tameez-GRPO-LoRA | its GRPO LoRA adapter | |
| Tiny-Aya-Fire-Tameez-GRPO-checkpoints | every Tiny Aya Fire GRPO checkpoint (steps 50–300) with optimizer states and logs | |
| Tiny-Aya-Fire-Hindi-SFT | Tiny Aya Fire after plain Hindi SFT (control), merged | |
| Tiny-Aya-Fire-Hindi-SFT-LoRA | its SFT LoRA adapter | |
| Llama-3.2-3B-Tameez-GRPO | Llama-3.2-3B after Hindi SFT + Tameez GRPO (step 100), merged | |
| Llama-3.2-3B-Tameez-GRPO-LoRA | its GRPO LoRA adapter | |
| Llama-3.2-3B-Tameez-GRPO-step300 | the last Llama GRPO checkpoint (not chosen), merged | |
| Llama-3.2-3B-Tameez-GRPO-checkpoints | every Llama GRPO checkpoint with optimizer states and logs | |
| Llama-3.2-3B-Hindi-SFT | Llama after plain Hindi SFT (control), merged | |
| Llama-3.2-3B-Hindi-SFT-LoRA | its SFT LoRA adapter and trainer checkpoints | |
| 📊 | tameez-hindi-safety-data | the curated SFT, RL and test splits, lexicons and Hinglish rewrites |
| 📊 | tameez-eval-outputs | every scored reply of every model and checkpoint, with every judge's verdict |
| 📊 | tameez-training-logs | per-step reward curves, sampled completions, RL rollouts, pipeline logs |
| 💻 | github.com/N0VA06/tameez | all code: data building, RL environment, rewards, training, evaluation, plots |
| 📝 | Article | the full write-up |
Collections: Tameez: models, LoRA adapters and artifacts · Tameez: datasets, logs, traces and benchmarks
Lexsi AI Safety and Alignment Hackathon
Tameez is a submission to the Lexsi AI Safety and Alignment Hackathon (Lexsi Labs × Cohere Labs × Hugging Face, November 2026): Track 2a, problem 4 · multilingual and code-mixed safety (primary) and Track 2b, problem 5 · safety-preserving fine-tuning.
- Cohere Labs: the policies are Tiny Aya Global (main) and its South Asia variant Tiny Aya Fire. Aya Expanse 8B generated the Hinglish training data and judged helpfulness. The Aya Dataset, Aya Red-teaming and the Aya Evaluation Suite supply SFT and test data.
- Lexsi Labs: both training stages run on AlignTune. Stage 1 calls
create_sft_trainer(LoRA, TRL backend); Stage 2 callscreate_rl_trainer(algorithm="grpo", loss_type="dr_grpo")with the six rewards passed through AlignTune's per-sample custom-reward contract. AlignTune writes alexsi_provenance.jsonrun record next to every adapter. Every stage's saved replies can be audited with AuditKIT (scripts/audit_with_auditkit.pyin the code repo: AuditKIT scorers, itsRefusalDetectorandcompare, CPU only). CuratorKIT re-checks the curated data splits for duplicates, train/test leakage and leaked secrets (scripts/curate_with_curatorkit.py): 3 of 18,642 training rows overlap a test prompt, all in benign tests.
License
Built with Llama. This model is a derivative of Llama 3.2 and is released under the
Llama 3.2 Community License. Llama 3.2 is licensed under the Llama 3.2
Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. Use must follow Meta's
Acceptable Use Policy. See NOTICE.
Citation
@misc{tameez2026,
title = {Tameez: Teaching 3B Models Manners in Hindi and Hinglish with GRPO},
author = {Vijay Srinivas P},
year = {2026},
url = {https://huggingface.co/IINOVAII/Llama-3.2-3B-Tameez-GRPO},
note = {Code: https://github.com/N0VA06/tameez}
}
- Downloads last month
- 240








