3 ms·
I've noted, over the years, many sites that scrape craigslist data for one purpose or another. They run for a few months, get popular, get noticed by craigslist
by synnik 17y ago
I've noted, over the years, many sites that scrape craigslist data for one purpose or another. They run for a few months, get popular, get noticed by craigslist, get shut down.
Do you have ongoing communications with them to avoid this fate?
- bluefish 17y agoAgreed. You really need to read the CL TOS and make sure you're working with CL to use their data.
- andrewljohnson 17y agoCould you name an example of this? I scraped CL for a while and they didn't say a word, and I've never heard of them shutting someone down.
- nuggien 17y agohow popular is your site? I remember they shut down some real estate site that mashed up craiglist data with google maps that got shut down pretty quickly. Not sure what the name was.
- catone 17y agoIf you visit http://crgslst.com/ http://crgslst.com/ -- a multi-city search site -- it pops up a message about how they were blocked. They also disabled Jeff Atwood's multi-city search: http://www.codinghorror.com/craigslist/ http://www.codinghorror.com/craigslist/ (he posted the letter he got from them) Then again, http://www.housingmaps.com/ http://www.housingmaps.com/ is still functional, and might be the most popular Craigslist mashup -- I'm assuming they have a deal in place with CL.
- weaksauce 17y agoFrom the TOU: http://www.craigslist.org/about/terms.of.use http://www.craigslist.org/about/terms.of.use u) use automated means, including spiders, robots, crawlers, data mining tools, or the like to download data from the Service - unless expressly permitted by craigslist; If you are serious about this site and you think that it could become a largish site you should definitely get permission.