4 ms·
Wow. That's a lot of bandwidth. I re-wrote the downloaders for siteadvisor a few years ago (hosted) and they didn't need anything close to that. They were ju
by darkarmani 13y ago
Wow. That's a lot of bandwidth. I re-wrote the downloaders for siteadvisor a few years ago (hosted) and they didn't need anything close to that. They were just downloading EXEs and other content from pages the crawlers had already seen, but still.
- adamseabrook 13y agoThose servers run our ElasticSearch index so move TB of data around inside the network and out to customers. Our actual crawlers could happily sip from a 5mbps pipe.