3 ms·
He using amazon cloud services , instead of 80 legs for the crawling. Wasn't the economics of 80 legs was supposed to be much better then amazon ?
by ippisl 17y ago
He using amazon cloud services , instead of 80 legs for the crawling.
Wasn't the economics of 80 legs was supposed to be much better then amazon ?
- jdrock 17y agoPete still uses us for his crawling. He switched over to AWS for some of his crawling because we had a throttle on some of the sites he wanted to crawl. But now that we know we can crawl those sites more quickly, we're looking into relaxing the throttle.