hikewa/dialectic-qwen2.5-1.5b-lora

Dialectic Reasoning LoRA — Qwen2.5-1.5B

Exploratory Qwen2.5-1.5B LoRA adapter fine-tuned on 205 public dialectic reasoning traces to improve integrative resolution under conflicting frames.

What It Does

Compared to the base Qwen2.5-1.5B-Instruct, this adapter produces responses that:

• Identify genuine tensions between competing perspectives
• Engage with the strongest form of each argument
• Synthesize insights rather than picking sides or hedging
• Handle greetings naturally (unlike the 0.5B variant)

Training Details

Parameter	Value
Base model	Qwen/Qwen2.5-1.5B-Instruct
Method	LoRA (r=16, alpha=32)
Target modules	q_proj, k_proj, v_proj, o_proj
Training examples	205
Epochs	3
Final eval loss	1.84
Training time	5.5 min (Apple MPS)
Dataset	hikewa/dialectic-reasoning-traces

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct", torch_dtype="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(base, "hikewa/dialectic-qwen2.5-1.5b-lora")
tokenizer = AutoTokenizer.from_pretrained("hikewa/dialectic-qwen2.5-1.5b-lora", trust_remote_code=True)

messages = [
    {"role": "system", "content": "You reason carefully through problems by considering competing perspectives."},
    {"role": "user", "content": "Should AI systems be transparent about their reasoning?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Model Family

Model	Eval Loss	Link
Qwen2.5-0.5B	2.19	hikewa/dialectic-qwen2.5-0.5b-lora
Qwen2.5-1.5B	1.84	hikewa/dialectic-qwen2.5-1.5b-lora
Qwen3-4B	1.32	hikewa/dialectic-qwen3-4b-lora
Qwen3-8B	1.26	hikewa/dialectic-qwen3-8b-lora

Position In The Model Family

This is a smaller exploratory variant. The strongest published evaluation currently belongs to the 8B model, which was trained on a larger internal 510-trace corpus rather than only the smaller public release used here.

Limitations

• 205 training examples is small — the model may repeat patterns
• Training traces were generated by Claude Sonnet, so the model inherits that reasoning style
• 1.5B is a mid-range model — handles dialectic structure well but less nuanced than the 4B
• This model should not be treated as evidence-equivalent to the 8B result