4 ms·

> These efforts to help keep the broader digital ecosystem safe supplement the protections we have to safeguard Android users on certified devices. We ensured G

> These efforts to help keep the broader digital ecosystem safe supplement the protections we have to safeguard Android users on certified devices. We ensured Google Play Protect, Android’s built-in security protection, automatically warns users and removes applications known to incorporate IPIDEA SDKs, and blocks any future install attempts. Nice to see Google Play Protect actually serving a purpose for once.

trollbridge 9mo ago
Yeah, it serves the purpose of blocking this kind of proxy traffic that isn't in Google's personal best interests. Only Google is allowed to scrape the web.
- a456463 9mo ago
  Yup exactly. Google must be the only one allowed to scrape the web. Google can't have any other competition. Calling it in "user's best interest" is just like their other marketing cons: "play integrity for user's security" etc
- viraptor 9mo ago
  Have you got any proof of Google scraping from residential proxies users don't know about, rather than from their clearly labelled AS? Otherwise you're mixing entirely different things into one claim.
  - idiotsecant 9mo ago
    I don't think parent post is claiming that Google is using other people's networks to scrape the web only that they have a strong incentive to keep other players from doing that.
    - viraptor 9mo ago
      No, there are other scrapers that Google doesn't block or interact with. You can even run scraping from GCP. This has nothing to do with "only Google is allowed to scrape". They even host apps which exist for scraping data, like https://play.google.com/store/apps/details?id=com.socialleadscraper https://play.google.com/store/apps/details?id=com.sociallead...
  - 4nonverbal-alit 9mo ago
    That's the whole point. Websites that try to block scraping attempts will let google scrape without any hurdle because of google's ads and search network. This gives google some advantage over new players because as a new name brand you are hardly going to convince a website to allow scraping even if your product may actually be more advantageous to the website (for example assume you made a search engine that doesn't suck like google, and aggregates links instead of copying content from your website). Proxies in comparison can allow new players to have some playing chance. That said I doubt any legitimate & ethical business would use proxies.
- vachina 9mo ago
  This is demonstrably false by the success of many scrapers from AI companies.
  - Nextgrid 9mo ago
    LLMs aren't a good indicator of success here because an LLM trained on 80% of the data is just as good as one trained on 100%, assuming the type/category of data is distributed evenly. Proxies help when you do need to get access to 100% of the data including data behind social media loginwalls.
- 1vuio0pswjnm7 9mo ago
  "Only Google is allowed to scrape the web." If I'm not mistaken, the plaintiffs in the US v Google antitrust litigation in the DC Circuit tried to argue that website operators are biased toward allowing Google to crawl and against allowing other search engines to do the same The Court rejected this argument because the plaintiffs did not present any evidence to support it For someone who does not follow the web's history, how would one produce direct evidence that the bias exists
  - SkiFire13 9mo ago
    > For someone who does not follow the web's history, how would one produce direct evidence that the bias exists Take a bunch of websites, fetch their robots.txt file and check how many allow GoogleBot but not others?
  - 1vuio0pswjnm7 9mo ago
    Common Crawl provides gzipped robots.txt collections
- miki123211 9mo ago
  Google does not use residential proxies. This does nothing against your ability to scrape the web the Google way, AKA from your own assigned IP range, obeying robots.txt, and with an user agent that explicitly says what you're doing and gives website owners a way to opt out. What Google doesn't want (and I don't think that's a bad thing) is competitors scraping the web in bad faith, without disclosing what they're doing to site owners and without giving them the ability to opt out. If Google doesn't stop these proxies, unscrupulous parties will have a competitive advantage over Google, it's that simple. Then Google will have to decide between just giving up (unlikely) or becoming unscrupulous themselves.
  - ryanjshaw 9mo ago
    > This does nothing against your ability to scrape the web the Google way I thought that Google has access to significant portions of the internet that non-Google bots won’t have access to?
    - morkalork 9mo ago
      Their crawler has known IPs that get a white-glove treatment by every site with a paywall for example
direwolf20 9mo ago
Does it also block unwanted traffic from Google apps or does it have a particular hatred for companies that interfere with Google's business model?
- tgsovlerkhgsel 9mo ago
  Play Protect blocks malicious apps, not network traffic, so no, it obviously doesn't interfere with Google's apps. AFAIK it also left SmartTube (an alternative YouTube client) alone until the developer got pwned and the app trojanized with this kind of SDK, and the clean versions are AFAIK again being left alone. No guarantee that it won't change in the future, of course, but so far they seem to not be abusing it.
  - direwolf20 9mo ago
    Does malicious mean interfering with Google's business model, or does it include intrusive advertising?
    - tgsovlerkhgsel 9mo ago
      Malicious here means "most people who aren't trying to argue semantics or otherwise be smartasses about it would consider it malware". That's why the example I gave is a semi-popular software the allows watching YouTube without ads without a premium subscription, i.e. at least in the case I observed, I don't believe this was weaponized against apps that interfere with their business model. As for "intrusive advertising is malicious", see the second part of the first sentence.
      - direwolf20 9mo ago
        I'm not being a smartass. Intrusive ads are malware. Adware used to be a category in virus scanners, then stopped when virus scanners wanted to run ads themselves. Proxying traffic is not malware, since it doesn't affect me in any way.
    - ThePowerOfFuet 9mo ago

trollbridge 9mo ago

Yeah, it serves the purpose of blocking this kind of proxy traffic that isn't in Google's personal best interests. Only Google is allowed to scrape the web.

a456463 9mo ago
Yup exactly. Google must be the only one allowed to scrape the web. Google can't have any other competition. Calling it in "user's best interest" is just like their other marketing cons: "play integrity for user's security" etc
viraptor 9mo ago
Have you got any proof of Google scraping from residential proxies users don't know about, rather than from their clearly labelled AS? Otherwise you're mixing entirely different things into one claim.
- idiotsecant 9mo ago
  I don't think parent post is claiming that Google is using other people's networks to scrape the web only that they have a strong incentive to keep other players from doing that.
  - viraptor 9mo ago
    No, there are other scrapers that Google doesn't block or interact with. You can even run scraping from GCP. This has nothing to do with "only Google is allowed to scrape". They even host apps which exist for scraping data, like https://play.google.com/store/apps/details?id=com.socialleadscraper https://play.google.com/store/apps/details?id=com.sociallead...
- 4nonverbal-alit 9mo ago
  That's the whole point. Websites that try to block scraping attempts will let google scrape without any hurdle because of google's ads and search network. This gives google some advantage over new players because as a new name brand you are hardly going to convince a website to allow scraping even if your product may actually be more advantageous to the website (for example assume you made a search engine that doesn't suck like google, and aggregates links instead of copying content from your website). Proxies in comparison can allow new players to have some playing chance. That said I doubt any legitimate & ethical business would use proxies.
vachina 9mo ago
This is demonstrably false by the success of many scrapers from AI companies.
- Nextgrid 9mo ago
  LLMs aren't a good indicator of success here because an LLM trained on 80% of the data is just as good as one trained on 100%, assuming the type/category of data is distributed evenly. Proxies help when you do need to get access to 100% of the data including data behind social media loginwalls.
1vuio0pswjnm7 9mo ago
"Only Google is allowed to scrape the web." If I'm not mistaken, the plaintiffs in the US v Google antitrust litigation in the DC Circuit tried to argue that website operators are biased toward allowing Google to crawl and against allowing other search engines to do the same The Court rejected this argument because the plaintiffs did not present any evidence to support it For someone who does not follow the web's history, how would one produce direct evidence that the bias exists
- SkiFire13 9mo ago
  > For someone who does not follow the web's history, how would one produce direct evidence that the bias exists Take a bunch of websites, fetch their robots.txt file and check how many allow GoogleBot but not others?
- 1vuio0pswjnm7 9mo ago
  Common Crawl provides gzipped robots.txt collections
miki123211 9mo ago
Google does not use residential proxies. This does nothing against your ability to scrape the web the Google way, AKA from your own assigned IP range, obeying robots.txt, and with an user agent that explicitly says what you're doing and gives website owners a way to opt out. What Google doesn't want (and I don't think that's a bad thing) is competitors scraping the web in bad faith, without disclosing what they're doing to site owners and without giving them the ability to opt out. If Google doesn't stop these proxies, unscrupulous parties will have a competitive advantage over Google, it's that simple. Then Google will have to decide between just giving up (unlikely) or becoming unscrupulous themselves.
- ryanjshaw 9mo ago
  > This does nothing against your ability to scrape the web the Google way I thought that Google has access to significant portions of the internet that non-Google bots won’t have access to?
  - morkalork 9mo ago
    Their crawler has known IPs that get a white-glove treatment by every site with a paywall for example

direwolf20 9mo ago

Does it also block unwanted traffic from Google apps or does it have a particular hatred for companies that interfere with Google's business model?

tgsovlerkhgsel 9mo ago
Play Protect blocks malicious apps, not network traffic, so no, it obviously doesn't interfere with Google's apps. AFAIK it also left SmartTube (an alternative YouTube client) alone until the developer got pwned and the app trojanized with this kind of SDK, and the clean versions are AFAIK again being left alone. No guarantee that it won't change in the future, of course, but so far they seem to not be abusing it.
- direwolf20 9mo ago
  Does malicious mean interfering with Google's business model, or does it include intrusive advertising?
  - tgsovlerkhgsel 9mo ago
    Malicious here means "most people who aren't trying to argue semantics or otherwise be smartasses about it would consider it malware". That's why the example I gave is a semi-popular software the allows watching YouTube without ads without a premium subscription, i.e. at least in the case I observed, I don't believe this was weaponized against apps that interfere with their business model. As for "intrusive advertising is malicious", see the second part of the first sentence.
    - direwolf20 9mo ago
      I'm not being a smartass. Intrusive ads are malware. Adware used to be a category in virus scanners, then stopped when virus scanners wanted to run ads themselves. Proxying traffic is not malware, since it doesn't affect me in any way.
  - ThePowerOfFuet 9mo ago