mytorch

A character-level GPT trained with mytorch, a small educational deep learning framework with its own autograd engine.

Training

Hyperparameter	Value
batch_size	256
context_len	64
dataset	tinystories
epochs	3
keep_last	3
log_every	100
lr	0.000369
n_embd	384
n_head	8
n_layer	10
sample_every	1000
save_every	2000
source	train
split_ratio	0.9
subset	None
vocab_size	174

Steps: 30000

Characters seen: 483,840,000

Last training loss: 0.7385 nats/char

Sample

Once upon a time, there was a little boy named Timmy. Timmy loved to eat pears for breakfast. One day, he went to the park with his mom and he saw a big rock. He followed the path, but it was too slippery. He dropped it and ran away. The ball flew away with the hole and it started to argue. They tried to get out, b

Usage

uv run python -m examples.GPT.hf download --repo mmczaplinski/mytorch --out examples/GPT/checkpoints/hf
uv run python -m examples.GPT.generate examples/GPT/checkpoints/hf --prompt "Once upon a time"

The checkpoint is a compressed .npz of the weights plus a sibling .json with the hyperparameters and the character vocabulary (extra.stoi).