2 ms·
It’s not that hard. I’ve done this. The list of URLs for a crawl is several hundred gigs. Easily fits in a lookup index on a single instance.
by binarymax 4mo ago
It’s not that hard. I’ve done this. The list of URLs for a crawl is several hundred gigs. Easily fits in a lookup index on a single instance.