3 ms·
The world's largest scraper is complaining about being scraped...when Yelp and all the rest of the local businesses directories realized they were being scraped
by ddemchuk 16y ago
The world's largest scraper is complaining about being scraped...when Yelp and all the rest of the local businesses directories realized they were being scraped and the data was being used to google's advantage, it doesn't appear that Google gave two shits. Tables have turned, and it's like someone stole the handball from the recess yard.
- SeanLuke 16y ago> when Yelp and all the rest of the local businesses directories realized they were being scraped and the data was being used to google's advantage, http://yelp.com/robots.txt http://yelp.com/robots.txt Not a mention of GoogleBot.
- ddemchuk 16y agowell, they're presented with either 1) killing their organic search traffic by blocking Google or 2) allowing the crawlers but not being able to stop their data from being aggregated and included in local results. Yelp's API specifically says their data cannot be aggregated with other sources. When presented with that, Google simply said they weren't using the API, they were scraping. Apparently scraping is a free pass to do whatever the hell you want if you're Google.