hynky

4 ms·

1. 1.▲
   by hynky 9mo ago

   HF recently released a blog about their pre-training dataset from PDFs. Alongside that they also shared a graph how much PDFs they were able to fetch by date, and the old internet seems to be mostly dead.
2. 2.▲
   by hynky 9mo ago

   HuggingFace recently released a report on their FinePDFs dataset, with this saddening report on state of current availability of old internet.
3. 3.▲
   FinePDFs: 3T token dataset made from internet PDFs

   3 points by hynky 1y ago|1 comments
4. 4.▲
   by hynky 1y ago

   yes, both core web datasets are publicly available as well as the rest
5. 5.▲
   FineWeb2: Adapting Pre-Training Data Processing to Every Language(arxiv.org)

   7 points by hynky 1y ago|0 comments
6. 6.▲
   FineWeb2 dataset: A sparkling update with 1000s of languages(huggingface.co)

   2 points by hynky 2y ago|0 comments