3 ms·
Do you still have the raw crawl data? If so, have you considered uploading it to the Internet Archive to help preserve it for the future?
by MrRadar 9y ago
Do you still have the raw crawl data? If so, have you considered uploading it to the Internet Archive to help preserve it for the future?
- benjojo12 9y agoNice idea, I've rolled up all of my data and crawl database and put it on the internet archive: https://archive.org/details/gopher-may-2017.tar https://archive.org/details/gopher-may-2017.tar
- MrRadar 9y agoThanks! It's amusing how the entirety of Gopher fits into a 1.6 GB archive compared to the petabytes (exabytes?) it would take to archive the entirety of the WWW.
- bastawhiz 9y agoThe internet is many, many, many exabytes large at this point. Hell, I'd bet YouTube alone is a few hundred exabytes at minimum.
- snakeanus 9y agoCould someone please help seed the torrent? It has been stuck at 99.9% for the past day.
- MrRadar 9y agoThe Internet Archive should be seeding all the torrents they host. Have you tried restarting your torrent client, forcing it to recheck your local data, or re-downloading and re-adding the torrent file?