3 ms·
80Legs: A Web Crawler as a Service
- hungdaddy 17y agocool
- timdorr 17y agoOur unique distributed architecture stitches together computers from all around the world to deliver exceptional crawling performance at an incredibly low cost. Sounds like they've got a botnet :| Of course, I'm sure it's nowhere near as insidious. But are they piggybacking this on top of some other distributed computing platform or service? Can I contribute (and maybe get paid for my time)? Edit: Do some basic research, Tim... http://gigaom.com/2009/05/14/80legs-cares-about-your-bandwidth-cap/ http://gigaom.com/2009/05/14/80legs-cares-about-your-bandwid...