Update README.md

README.md

@@ -1,3 +1,39 @@
----
-license: mit
----
+---
+license: mit
+---
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+An unofficial reproduced PRepBN-Llama-350M checkpoints for [SLAB](https://github.com/xinghaochen/SLAB/).
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [https://github.com/xinghaochen/SLAB/]
+- **Paper [optional]:** [https://arxiv.org/abs/2405.11582]
+
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+https://github.com/xinghaochen/SLAB/tree/main/llama
+
+```
+python evaluation.py --ckpt <checkpoint-path>
+```
+
+[Results](https://github.com/xinghaochen/SLAB/blob/main/docs/llama.png)
+
+
+**BibTeX:**
+
+```
+@inproceedings{guo2024slab,
+  title={SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization},
+  author={Guo, Jialong and Chen, Xinghao and Tang, Yehui  and Wang, Yunhe},
+  booktitle={International Conference on Machine Learning},
+  year={2024}
+}
+```