PoliteChainsaw/Lewi-Surge-Preview-3B-Chat-Early

Lewi Surge Preview 3B (Chat, Early)

An early, partially fine-tuned chat checkpoint of Lewi Surge, an open mixture-of-experts language model for programming trained from scratch by one person in Ukraine.

⚠️ Read this first. This checkpoint went through less than a quarter of the planned chat fine-tuning (SFT). The run had to stop when the compute budget ran out. It already answers in the chat format and stops on its own, but it is not a finished assistant. We publish it as it is, so you can see the real state of the project.

💛 Why it's unfinished, and how you can help. Lewi is built without funding, on free and donated compute. The full fine-tuning (identity, instruction following, code and multilingual conversations) is prepared and ready to run; what's missing is GPU time. About $3 buys one hour on an NVIDIA GH200, and just $50 would unlock ~120 hours on a GH200 (the provider gives us access after a $50 deposit, and our free credits cover the rest), enough to finish SFT and fix the problems below. If you'd like to see Lewi finished, you can support it on Ko-fi.

Known problems

All of these come from the fine-tuning being cut short, not from the architecture. The data to fix them is already prepared (382k conversations, including identity and instruction-following sets).

Problem	What you'll see	Fix (needs compute)
Weak identity	May claim to be "a software engineer" or another persona instead of Lewi	Identity conversations in the full SFT
Ignores format and language instructions	"Only code" or "answer in Ukrainian" is often not followed	Instruction-following data in the full SFT
Loops in long lists	Long enumerations can repeat items	More SFT steps + long-context stage
Short context	Trained on up to 16k-token conversations; 128k stage is in progress	Long-context stage on TPU (queued)
Factual errors	States wrong facts confidently, especially outside code	Larger model and more data (Lewi Surge 24B2A)

Model

Base	Lewi-Surge-Preview-3B-Base (HumanEval 25.0%, MBPP 29.6% as a base model)
Parameters	~3.45B total, ~0.5B active per token
Layers	16 = 4 × (3 Gated DeltaNet + 1 full attention)
Experts	128 per layer, top-6 routed + 1 shared
Chat format	ChatML (<|im_start|> / <|im_end|>), no system prompt required, no thinking mode

Usage

Requires transformers >= 5.0.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "PoliteChainsaw/Lewi-Surge-Preview-3B-Chat-Early"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "Write a Python function that checks if a string is a palindrome."}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=256, do_sample=False, repetition_penalty=1.1)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

A light repetition_penalty (1.05–1.1) helps with the looping issue. Runs on a single consumer GPU (~7 GB in bf16).

Limitations

Early checkpoint of a small model: expect wrong facts, uneven instruction following and weak non-Python code. Not safety-tuned. Do not use it for anything where mistakes matter.

Links

Project page · Code · Support on Ko-fi