4 ms·

> I don't like working on projects that are actively pitting me against someone though. I'm surprised you didn't also didn't mention the ethical problems with

• patio11 12y ago
  Google doesn't exactly tie itself in knots that they built a few hundred billion dollars on top of "We're going to crawl the entire Internet and datamine it for our own purposes. Nobody will agree to this, so we won't ask them. Instead, we will offer easy ways to opt out after having achieved hegemonic control of Internet navigation."
  • eli 12y ago
    Though to be fair, that's how all the pre-Google search engines worked too.
  • jonas21 12y ago
    There have been easy ways to opt out since day one. Google has always respected robots.txt.
• ixmatus 12y ago
  Meh. I didn't have any qualms with that, per-se. As others have pointed out I think there are ethical concerns with what Google is doing itself. Also it's indicative of something if people are building businesses to scrape something they could offer as a pay-for API that many people would gladly / happily pay lots of money for. The arguments for NOT doing that are ridiculous because people will get the data regardless.