4 ms·

As the post says, Google only scrapes the websites that want to be scraped. Sure, it's opt-out (via robots.txt) rather than opt-in, but they do give you a choic

As the post says, Google only scrapes the websites that want to be scraped. Sure, it's opt-out (via robots.txt) rather than opt-in, but they do give you a choice. You can even decide between no scraping at all and opting out on a per-scraper basis, and Google will absolutely honor your preferences in that regard. SERP API just assumes everybody wants to be scraped, and doesn't give you a choice. (whether websites should have such a choice is a different matter entirely).

Nextgrid 10mo ago
This is a bad argument because Google is using its monopoly to effectively force websites to allow Google to scrape them.
lurking_swe 10mo ago
requiring me to explicitly opt-out of something is NOT the same thing as getting my consent. So your argument breaks down there. You know what getting my consent would look like? Google hosting a form where i can tell them PLEASE SCRAPE MY WEBSITE and include it in your search results. That is what consent looks like. Google has never asked for my consent. Yet they expect others to behave by different rules. Now where google may have a reasonable case is that google scrapes with the intention of offering the data “for free”. SerpAPI does not.
- halJordan 10mo ago
  It's never been the case that if you put something into public, then you get to reserve your right to refuse public access. Either it's public and strangers can look at it. Or it's private and you need to implement a gate.
- carlosjobim 9mo ago
  > You know what getting my consent would look like? Google hosting a form where i can tell them PLEASE SCRAPE MY WEBSITE and include it in your search results. That is what consent looks like. Just for anybody wondering, they have always had such a form as well. Apart from their general crawling.
gnfargbl 10mo ago
If this is about protecting third parties from being scraped, why does Google have an interest at all? Surely Google won't have the relevant third-party data itself because, as you say, Google respects robots.txt. So how can that data be scraped from Google? I don't think this suit is actually about that, though. I think Google's complaint is that > SerpApi deceptively takes content that Google licenses from others In other words, this is just a good old-fashioned licence violation.

Nextgrid 10mo ago

This is a bad argument because Google is using its monopoly to effectively force websites to allow Google to scrape them.

lurking_swe 10mo ago

requiring me to explicitly opt-out of something is NOT the same thing as getting my consent. So your argument breaks down there. You know what getting my consent would look like? Google hosting a form where i can tell them PLEASE SCRAPE MY WEBSITE and include it in your search results. That is what consent looks like. Google has never asked for my consent. Yet they expect others to behave by different rules. Now where google may have a reasonable case is that google scrapes with the intention of offering the data “for free”. SerpAPI does not.

halJordan 10mo ago
It's never been the case that if you put something into public, then you get to reserve your right to refuse public access. Either it's public and strangers can look at it. Or it's private and you need to implement a gate.
carlosjobim 9mo ago
> You know what getting my consent would look like? Google hosting a form where i can tell them PLEASE SCRAPE MY WEBSITE and include it in your search results. That is what consent looks like. Just for anybody wondering, they have always had such a form as well. Apart from their general crawling.

gnfargbl 10mo ago

If this is about protecting third parties from being scraped, why does Google have an interest at all? Surely Google won't have the relevant third-party data itself because, as you say, Google respects robots.txt. So how can that data be scraped from Google? I don't think this suit is actually about that, though. I think Google's complaint is that > SerpApi deceptively takes content that Google licenses from others In other words, this is just a good old-fashioned licence violation.