---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
base_model: Qwen/Qwen2.5-1.5B-Instruct
language:
  - en
tags:
  - qwen2.5
  - qlora
  - peft
  - lora
  - semiconductor
  - technical-writing
  - product-brief
  - datasheet
  - user-guide
metrics:
  - loss
  - accuracy
model-index:
  - name: Qwen2.5-1.5B-TechWriter-Instruct
    results:
      - task:
          type: text-generation
          name: Causal language modeling (teacher-forced eval)
        dataset:
          name: semiconductor technical-writing ChatML mix (10% holdout)
          type: json
          split: test
        metrics:
          - type: loss
            value: 0.1404
            name: Eval loss (epoch 3)
          - type: accuracy
            value: 0.9488
            name: Mean token accuracy (epoch 3)
---

# Qwen2.5-1.5B TechWriter (merged)

Merged QLoRA fine-tune of [`Qwen/Qwen2.5-1.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) for **semiconductor and data-center interconnect** technical-marketing and documentation style: product briefs, datasheets, application notes, and user-guide CLI sections.

This is the **inference repo**. Load it with `AutoModelForCausalLM` or `pipeline("text-generation")`.

This is an unofficial specialist checkpoint trained on a private mix of extracted vendor PDFs plus cleaned synthetic docs. It will still invent SKUs if you ask it to write a brief for a product that was never in the gold data.

Adapter-only (smaller download, resume / compose):

**`Shankarblr/Qwen2.5-1.5B-TechWriter-LoRA`**

## What it is for

- Product briefs, datasheet feature lists, application notes
- Host-adapter / switch CLI user-guide sections
- Spec extraction and short grounded QA over a pasted excerpt
- Internal draft generation that should stay on-genre and internally consistent
  (one process node, one primary throughput, one form factor)

Not for:

- Authoritative datasheet numbers without a human / source check
- Legal, safety, or customer-facing specs shipped as-is
- Languages other than English
- General chat outside semiconductor interconnect / storage / DPU content

## Load and run

Use the **Qwen2.5 Instruct chat template**. Do not hand-roll ChatML if the tokenizer already has a template.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct"

tokenizer = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForCausalLM.from_pretrained(
    REPO,
    torch_dtype=torch.float16,
    device_map="auto",
)

messages = [
    {
        "role": "system",
        "content": (
            "You are a technical marketing and documentation writer for semiconductor "
            "and data-center interconnect products. Write clear, structured content. "
            "Match the requested document type. Keep specifications internally consistent: "
            "one process node, one primary throughput, and one form factor unless the "
            "source explicitly lists options. Do not invent conflicting SKUs or CLI syntax."
        ),
    },
    {
        "role": "user",
        "content": "Draft a product brief covering a 40/50/100GbE converged network adapter series.",
    },
]

prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(
    **inputs,
    max_new_tokens=1024,
    do_sample=False,
    pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

### Pipeline

```python
from transformers import pipeline, AutoTokenizer

REPO = "Shankarblr/Qwen2.5-1.5B-TechWriter-Instruct"
tok = AutoTokenizer.from_pretrained(REPO)

pipe = pipeline(
    "text-generation",
    model=REPO,
    tokenizer=tok,
    max_new_tokens=1024,
    do_sample=False,
)

prompt = tok.apply_chat_template(
    [
        {"role": "system", "content": "You are a technical marketing and documentation writer for semiconductor and data-center interconnect products."},
        {"role": "user", "content": "For an enterprise 1/10/25GbE Ethernet switch series, author the Key Features portion of the product brief."},
    ],
    tokenize=False,
    add_generation_prompt=True,
)
print(pipe(prompt, return_full_text=False)[0]["generated_text"])
```

If you see `Both max_new_tokens and max_length(=20)`, the shipped `generation_config.json` still has a leftover `max_length`. Prefer passing **only** `max_new_tokens` at generate time, or edit that file before upload.

## Training

| Item | Value |
|---|---|
| Base | `Qwen/Qwen2.5-1.5B-Instruct` |
| Method | QLoRA 4-bit NF4 + double quant, then merged to fp16 |
| Trainer | Hugging Face TRL `SFTTrainer` / `SFTConfig` |
| Data | Private semiconductor technical-writing ChatML mix (6,765 rows) |
| Split | 90 / 10, seed 42 |
| Sequence length | 2,048 |
| LoRA | `r=16`, `alpha=32`, dropout 0.05 |
| Targets | `q_proj k_proj v_proj o_proj gate_proj up_proj down_proj` |
| LR / schedule | 2e-4 cosine, warmup 0.03 |
| Batch | 4 × grad accum 4 |
| Epochs / steps | **3 / 1,143** |
| Precision | fp16 or bf16 on CUDA (QLoRA compute dtype) |
| Wall time | **5,378 s ≈ 1 h 30 min** (1,143 steps, 4.70 s/it) |
| Train tokens seen | ~7.9M by end of epoch 3 |

### Eval (teacher-forced next token, not open generation)

| Checkpoint | Eval loss | Mean token accuracy | Eval entropy |
|---|---:|---:|---:|
| Epoch 2 | 0.1555 | 0.9453 | 0.1795 |
| **Epoch 3 (published)** | **0.1404** | **0.9488** | **0.1566** |

Mean train loss over the full run: **0.3846** (early epochs are higher; late-epoch train batches sit around 0.12–0.14). Grad norms stayed small (~0.07–0.14). Cosine LR decayed from ~5e-5 at epoch 2.02 to ~6e-9 at the last step.

**Read token accuracy correctly.** 94.9% is “the model assigned the gold next token” on the held-out ChatML strings. It is **not** factual accuracy on silicon SKUs, and it is not a win-rate against a human editor.

## Dataset mix

6,765 rows after dropping dirty synthetic gold (multi-node, multi-throughput, grammar doubles, stubs).

| Origin | Rows |
|---|---:|
| `synthetic_cleaned` | 4,757 |
| `extracted` (real vendor PDFs) | 1,930 |
| `synthetic_consistent` | 78 |

| Task | Rows |
|---|---:|
| generate | 2,532 |
| spec_json | 1,099 |
| cli_extract_syntax | 880 |
| cli_multiturn_syntax | 880 |
| grounded_qa | 544 |
| extract_specs / outline / multi_turn_section | 248 each |
| cli_when_to_use | 44 |
| multi_turn_consistent_specs | 42 |

Doc types: user_guide 2,772 · product_brief 2,528 · datasheet 613 · application_note 305 · technology_brief 229 · white_paper 222 · competitive_report 96.

Extracted PDFs cover converged network adapters, Ethernet switch silicon, host-adapter CLI, storage / fabric features, and related interconnect material. A large share of *generate* gold is still synthetic house-style prose, so the model can emit invented series names if the prompt does.

## Intended prompt style

The train-time system prompts were:

1. **Writer** — semiconductor / interconnect house style; one process node / throughput / form factor.
2. **Editor** — revise copy; do not add contradictory specs.
3. **Product specialist** — answer only from the pasted excerpt; otherwise say you cannot determine it.

Match those at inference. Grounded QA quality collapses if you drop the specialist system prompt.

## Limitations

- 1.5B parameters. Long, consistent datasheets still drift.
- Synthetic SKUs in the pre-clean pool taught a habit of inventing product families. The recommended set drops the worst contradictions; it does not delete every invented family.
- Extracted CLI gold is the reliable part. Treat generated flags and identifiers as drafts.
- No DPO / GRPO on this published checkpoint (those datasets exist separately).
- Eval is teacher-forced loss/accuracy only.

## Files to upload (merged repo)

| File | Role |
|---|---|
| `model.safetensors` | Merged Qwen2.5-1.5B + LoRA (fp16) |
| `config.json` | Architecture |
| `generation_config.json` | Prefer `max_new_tokens` only; remove stray `max_length: 20` |
| `tokenizer.json` / `tokenizer_config.json` / `vocab.json` / `merges.txt` | Qwen tokenizer |
| `README.md` | This card |

Do **not** upload `checkpoint-*`, `optimizer.pt`, `rng_state.pth`, or the 4-bit train-time weights.

## Related

- Base: [`Qwen/Qwen2.5-1.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct)
- Method: QLoRA (NF4) + PEFT LoRA, TRL SFT
- Adapter: [`Shankarblr/Qwen2.5-1.5B-TechWriter-LoRA`](https://huggingface.co/Shankarblr/Qwen2.5-1.5B-TechWriter-LoRA)
- Sister run (same data/recipe): [`Shankarblr/Llama-3.2-3B-TechWriter-Instruct`](https://huggingface.co/Shankarblr/Llama-3.2-3B-TechWriter-Instruct)

## License

Apache 2.0, same family as the Qwen2.5-1.5B Instruct weights. This checkpoint is an unofficial style model and is not affiliated with any semiconductor vendor.