3 ms·
Maybe the crawl could be distributed somehow, and you could pull versions of the web from those distributed nodes via BitTorrent.
by ericd 8y ago
Maybe the crawl could be distributed somehow, and you could pull versions of the web from those distributed nodes via BitTorrent.