4 ms·
Is the training data public as well?
by nicce 3y ago
Is the training data public as well?
- raxxorraxor 3y agoThis time Apple could say that they didn't release it for privacy reasons and this time it would be correct for once.
- GeekyBear 3y agoYes > RedPajama-V2 is an open dataset for training large language models. The dataset includes over 100B text documents coming from 84 CommonCrawl snapshots and processed using the CCNet pipeline. https://github.com/togethercomputer/RedPajama-Data https://github.com/togethercomputer/RedPajama-Data > Dolma Dataset: an open dataset of 3 trillion tokens from a diverse mix of web content, academic publications, code, books, and encyclopedic materials. https://allenai.github.io/dolma/ https://allenai.github.io/dolma/