3 ms·
Which shows just how much Google profits from their virtual monopoly. If content companies knew that Google would make billions of dollars selling advertisement
by coliveira 14y ago
Which shows just how much Google profits from their virtual monopoly. If content companies knew that Google would make billions of dollars selling advertisement based on the content that they generated, it would be very hard for Google to pull the massive Internet scrapping that they managed to do from day one. It is interesting that people understand the illegitimacy of taking data from a single content source (Craigslist), but fail to see how unethical is to take data from everyone and sell advertisements based on that.
- nostromo 14y agoInterestingly, Craigslist is being consistent when it comes to not allowing scrapers, including Google. http://www.craigslist.org/robots.txt http://www.craigslist.org/robots.txt edit: oops! they're not actually blocking google from the apartment listings. thanks smackfu.
- smackfu 14y agoAren't they only disallowing particular pages there? Like apartments are at *.craigslist.org/apa/number.html which doesn't seem blocked by that robots.txt.