4 ms·
Yes > RedPajama-V2 is an open dataset for training large language models. The dataset includes over 100B text documents coming from 84 CommonCrawl snapshots an
by GeekyBear 3y ago
Yes
> RedPajama-V2 is an open dataset for training large language models. The dataset includes over 100B text documents coming from 84 CommonCrawl snapshots and processed using the CCNet pipeline.
https://github.com/togethercomputer/RedPajama-Data https://github.com/togethercomputer/RedPajama-Data
> Dolma Dataset: an open dataset of 3 trillion tokens from a diverse mix of web content, academic publications, code, books, and encyclopedic materials.
https://allenai.github.io/dolma/ https://allenai.github.io/dolma/