2 ms·
I get that but I'm not sure I see the author's point though. Replicating Archive.org isn't hard, it's pretty standard web spider stuff, it's just massively reso
by sh1mmer 18y ago
I get that but I'm not sure I see the author's point though. Replicating Archive.org isn't hard, it's pretty standard web spider stuff, it's just massively resource intensive.
You can try and distribute that but I suspect it would just end up like most Bit Torrent trackers where the head of the long tail has lots of support and the tail has little to no support.