3 ms·
Unfortunately things like a 500gb dump of sacredtexts website is not often found on torrent trackers or other 'warez' sites. Anna's is pretty great for written
by kelsey98765431 2y ago
Unfortunately things like a 500gb dump of sacredtexts website is not often found on torrent trackers or other 'warez' sites. Anna's is pretty great for written material that has been published offline, but even the wayback machine and archive.org have limited full scrapes aside from the published ones by the kiwix team.
- sigmoid10 2y agoSuch megadumps are only really interesting for people who train LLMs anyways. No way you can ever consume that for personal development. Individual sources are much more prevalent and useful to everyday life.
- mewpmewp2 2y agoIt would be interesting for a personal project to do some search and summarisation project, e.g. with RAG. Get the data, make embeddings, categorize and tag the data, then do search and have LLM summarize findings.
- kelsey98765431 2y ago:)