3 ms·
If you’re looking at The Pile, you also might consider the Red Pajama dataset. A new cleaned version was released recently https://www.cerebras.net/blog/slimpaj
by cschmidt 3y ago
If you’re looking at The Pile, you also might consider the Red Pajama dataset. A new cleaned version was released recently https://www.cerebras.net/blog/slimpajama-a-627b-token-cleaned-and-deduplicated-version-of-redpajama https://www.cerebras.net/blog/slimpajama-a-627b-token-cleane...
- CamperBob2 3y agoIs there a straightforward way to download that dataset, the way there was for the original RedPajama data? SlimPajama appears to have been released as 60,000 small files, which is ridiculous.