3 ms·
There's a thumping amount of data there. 485 torrents, each torrent containing 100 zips of around 500MB each, each zip containing 1000 papers. Total = 500MB *
by femto 11y ago
There's a thumping amount of data there. 485 torrents, each torrent containing 100 zips of around 500MB each, each zip containing 1000 papers. Total = 500MB * 100 * 485 ~ 25TB. If your have a 10Mbit/s link into your house, it's 280 days of continuous downloading.
New torrents seem to be appearing at the rate of 14 per month = 700GB/month = 2.7Mbit/s. For the majority of people, I'd say it is beyond any one person being able to mirror the lot.
It seems like an application for a Freenet type system, but one where the files are guaranteed to never be deleted, even if they are rarely accessed.
- harperlee 11y agoI would assume that most of that is due to PDFs weigthing a lot more than their content. If someone wants to donate IT skills and coding time (this is a thing that I've read several times in HN), perhaps it would be great that someone puts together a system that takes a PDF, OCRs it when necessary, detects images and graphs, and uses all of that to create a light version of the document (html/latex/markdown/whatever); and then puts all of those TB through that tool. The reduction in storage and network costs might be a lot more than a standard donation!
- kwhitefoot 11y agoIf enough people keep the torrents alive no one person would need to mirror all of it, certainly not all at once.