Update model card for checkpoints/checkpoint-02B

README.md

@@ -16,10 +16,6 @@ datasets:
 This repository stores milestone checkpoints for `gdn2_kla_1.3B` from the
 `gdn2_kla_1.3B_fineweb_edu_10bt` run.
 
-## Code
-
-- GitHub: https://github.com/gyunggyung/Gated_Linear_Attention2
-
 ## What This Model Is
 
 `gdn2_kla_1.3B` is a recurrent-only linear attention experiment. It starts from
@@ -40,6 +36,10 @@ k_t(\lambda_t w_t \odot v_t)^\top
 It is not a standard Transformers checkpoint and does not use softmax attention
 or SWA layers.
 
+## Code
+
+- GitHub: https://github.com/gyunggyung/Gated_Linear_Attention2
+
 ## License
 
 The model weights in this Hugging Face repository are released under Apache-2.0.
@@ -56,9 +56,9 @@ model repository is intended to be used with the standalone Apache-2.0 runtime.
 - Candidate: Kaczmarz-normalized GDN-2 gates
 - Training data source: FineWeb-Edu `sample/100BT` local parquet
 - Token budget for this run: 10B
-- Current milestone: 1,000,000,000 tokens
+- Current milestone: 2,000,000,000 tokens
 - Sequence length: 4096 tokens
-- Global batch tokens: 524,288
+- Global batch tokens: 1,048,576
 - Tokenizer: `TinyLlama/TinyLlama_v1.1`
 - Data shuffle seed: `3407`
 - Data shuffle buffer: `100000`