4 ms·
Craigslist reportedly blocks search engines to kill competitors
- paulgb 14y agoAs a PadMapper user this saddens me, but I can only wonder why they didn't do it sooner. Craigslist posts are so short-lived, I don't know why they would want them indexed.
- sigmadelta 14y agoDid these people do any fact checking? I just ran another test: visit http://sfbay.craigslist.org/bia/ http://sfbay.craigslist.org/bia/ spot ad for a kids Giant MTX 250 bike http://sfbay.craigslist.org/nby/bik/3189980073.html http://sfbay.craigslist.org/nby/bik/3189980073.html perform a google search for the same thing http://www.google.com/search?q=site%3Acraigslist.org+giant+mtx+san+francisco&btnG=Search http://www.google.com/search?q=site%3Acraigslist.org+giant+m... result number two in the set is that posting, with the following summary: Giant MTX 250 Kid's Mountain Bike sfbay.craigslist.org/nby/bik/3189980073.html 4 minutes ago ... Giant kid sized hardtail mountain bike for sale. It is a 21 speed, and it has front suspension. The bike was always taken care of, and is in perfect ... "4 minutes ago" tells me Google is having no problems hitting craigslist.
- danso 14y ago...Do you do any fact checking? I just checked the source: <meta name="robots" content="NOARCHIVE,NOFOLLOW">
- rcollamore 14y agoNothing new. That tag has always been in place. It's to prevent using an ad for SEO purposes
- sigmadelta 14y agoThose tags aren't as obvious in interpretation as they first appear. A full write-up is a http://noarchive.net/meta/ http://noarchive.net/meta/ for completeness, but in short: NOARCHIVE: ok to index this page, but don't cache the result permanently NOFOLLOW: don't follow any links on this page Note that these two things don't prevent a search engine from visiting a page, indexing it, and providing search results for it.