3 ms·
libgen is far from an archive of "most" books and publications, not even close. The most recent numbers from libgen itself are 2.4 million non-fiction books an
by cratermoon 2y ago
libgen is far from an archive of "most" books and publications,
not even close.
The most recent numbers from libgen itself are 2.4 million non-fiction books and 80 million science journal articles.
The Atlantic's database published in 2025 has 7.5 million books.[0]
The publishing industry estimates that many books are published each year.
As of 2010, Google counted over 129 million books[1].
At best an LLM like Llama will have have 20% of all books in its training set.
0. https://www.theatlantic.com/technology/archive/2025/03/libgen-meta-openai/682093/ https://www.theatlantic.com/technology/archive/2025/03/libge...
1. https://booksearch.blogspot.com/2010/08/books-of-world-stand-up-and-be-counted.html https://booksearch.blogspot.com/2010/08/books-of-world-stand...
- UltraSane 2y agoOn libgen.mx they claim to have 33,569,200 books and 84,844,242 articles
- cratermoon 2y agoStill an order of magnitude short of "all", and falling farther behind every year.