The argument hinges on Google getting access to Craigslist listings while others are denied. However, that doesn't seem to be true: http://www.craigslist.org/

The argument hinges on Google getting access to Craigslist listings while others are denied. However, that doesn't seem to be true: http://www.craigslist.org/robots.txt http://www.craigslist.org/robots.txt https://www.google.com/search?q=site%3Aseattle.craigslist.com https://www.google.com/search?q=site%3Aseattle.craigslist.co... Update: The comment you quoted is a year old and Craigslist only recently blocked Google. http://www.seomoz.org/blog/craigslist-blocks-search-engines http://www.seomoz.org/blog/craigslist-blocks-search-engines Perhaps they decided to block everyone to bolster a legal case against aggregators like Padmapper.