3 ms·
Sorry if I wasn't clear, at Blekko we had an index at our peak of 5 billion documents. And we had the full stack to crawl, rank, cleanse, and search that index
by ChuckMcM 9y ago
Sorry if I wasn't clear, at Blekko we had an index at our peak of 5 billion documents. And we had the full stack to crawl, rank, cleanse, and search that index as well. The crawl frontier was about 300 billion URIs. That was what I meant by "holding".