HuggingFaceFW/ablation-exp-textext-warc_trafilatura-28BT
Text Generation • 2B • Updated • 19 • 1
AI & ML interests
We release large pre-training datasets to accelerate open LLM development. Part of the Hugging Face Science team (hf.co/science)
Recent Activity
View all activity
Papers
How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data
FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
HuggingFaceFW/ablation-exp-dedup-ind_mh-global_line-350BT
Text Generation • 2B • Updated • 13
HuggingFaceFW/ablation-exp-dedup-ind_mh-global_3line-350BT
Text Generation • 2B • Updated • 15
HuggingFaceFW/ablation-exp-fw-base_filt-ind_mh-c4_filters-350BT
Text Generation • 2B • Updated • 14
HuggingFaceFW/ablation-exp-dedup-ind_mh-global_line_minwords-350BT
Text Generation • 2B • Updated • 19
HuggingFaceFW/ablation-exp-dedup-independent_minhash-350BT
Text Generation • 2B • Updated • 17
HuggingFaceFW/ablation-exp-fw-base_filtering-350BT
Text Generation • 2B • Updated • 15
HuggingFaceFW/ablation-exp-dedup-global_minhash-350BT
Text Generation • 2B • Updated • 20
HuggingFaceFW/ablation-model-redpajama2
Text Generation • 2B • Updated • 27
HuggingFaceFW/ablation-model-the-pile
Text Generation • 2B • Updated • 58 • 1
HuggingFaceFW/ablation-model-slimpajama
Text Generation • 2B • Updated • 39 • 2
HuggingFaceFW/ablation-model-dolma-v1_6
Text Generation • 2B • Updated • 29 • 2
HuggingFaceFW/ablation-model-c4
Text Generation • 2B • Updated • 61 • 4
HuggingFaceFW/ablation-model-refinedweb
Text Generation • 2B • Updated • 46 • 3
HuggingFaceFW/ablation-model-fineweb-v1
Text Generation • 2B • Updated • 143 • 14