12 ms·
How do startups get around the Craigslist TOU against using "automated means, including spiders, robots, crawlers, data mining tools, or the like to download da
by ilamont 15y ago
How do startups get around the Craigslist TOU against using "automated means, including spiders, robots, crawlers, data mining tools, or the like to download data from the Service" (http://www.craigslist.org/about/terms.of.use http://www.craigslist.org/about/terms.of.use)?
- paulhauggis 15y agoI wondered the same. A guy was actually sued over this for his automated crawling/posting bots he was selling as a service. From what I've heard, their lawyers are pretty relentless. I think he is also a reader on HN.
- jpdoctor 15y agoDon't the padmapper/etc housing crawlers do this? Those are incredibly useful too, it is much easier to see those things on a map.
- CoffeeDregs 15y agoYou don't crawl Craigslist; you tickle Google. Craigslist can't block them... But you'll still need an excellent lawyer...
- slig 15y agoI thought Google was heavily against automated searches and blocked bots on spot.
- gms 15y agoThey don't - it's a bit "don't ask, don't tell". The hope is that if Craigslist ever decides to enforce the TOU, said startup is popular enough that they can work out a deal.
- binarysolo 15y agoI think a solution using RSS feeds may get around this, if you can have RSS feeds to everything. (Not super familiar with CL feeds.)