ImperiumOfMan

5 ms·

1. 1.▲
   New AI Books5 Dataset for training LLMs further (July 2024 update)(twitter.com)

   4 points by ImperiumOfMan 2y ago|1 comments
2. 2.▲
   by ImperiumOfMan 2y ago

   - More than 600,000 fiction and non-fiction book full-texts - More than 8,000,000 scholarly publications, magazines, and manuals full-texts - More than 5,000,000 US patents - 164,000,000 metadata records Format is described at https:/
3. 3.▲
   AI Libstc3 (July 2024) Dataset for training LLMs further(twitter.com)

   2 points by ImperiumOfMan 2y ago|1 comments
4. 4.▲
   by ImperiumOfMan 2y ago

   What? - More than 600,000 fiction and non-fiction book full-texts - More than 8,000,000 scholarly publications, magazines, and manuals full-texts - More than 5,000,000 US patents - 164,000,000 metadata records Format Zstd compressed file,
5. 5.▲
   AI Dataset Libstc2 a.k.a. Books4(old.reddit.com)

   2 points by ImperiumOfMan 2y ago|1 comments
6. 6.▲
   by ImperiumOfMan 2y ago

   More than 400,000 fiction and non-fiction book full-texts. Multiple languages, curated, deduplicated. More than 6,000,000 scholarly publications, magazines, and manuals full-texts. Multiple languages, curated, deduplicated. 150,000,000 meta
7. 7.▲
   by ImperiumOfMan 2y ago

   Origin of the data: - LibGen collections gathered over decades - Sci-Hub - Additional fresh scholar papers and books collected by Library STC
8. 8.▲
   AI Books4 Dataset for training LLMs further(old.reddit.com)

   46 points by ImperiumOfMan 2y ago|48 comments
9. 9.▲
   Search over Sci-Hub Content(libstc.cc)

   2 points by ImperiumOfMan 2y ago|0 comments