5 ms·

There are many arguments against this but not many brought the implications for search engines. If websites implement this, it will effectively make building a

by devsda 3y ago

There are many arguments against this but not many brought the implications for search engines. If websites implement this, it will effectively make building a web search engine impossible for new entrants. The current players can whitelist/attest their own clients while categorizing every other scraping clients as bots. If not for other reasons, I can't see how Google a search company can be allowed to push something that can kill competition using its market dominance in other areas like browsers.

justcool393 3y ago
> If not for other reasons, I can't see how Google a search company can be allowed to push something that can kill competition using its market dominance in other areas like browsers. Because antitrust has been dead for a while. Chrome is a tool to drive people to Google and Google ads and nothing more. I will say, I did appreciate Microsoft having a browser engine with IE and Edge, even if the former was notoriously a pain, it gave competition in the space. Unfortunately, that's not the case anymore and everything is either Chrome (Blink), Firefox (Gecko), or Safari (WebKit). And it's pretty clear what Chrome has done once that have amassed a dominant market share. I'm sure there are Googlers who think they're legitimately making the web a safer place, but I think the real reason is pretty clear if you take a birds eye view.
- deleted 3y ago
  [deleted]
- retrac 3y ago
  Unfortunately it seems that nearly all software platforms with a dominant market share, end up degenerating to serve only one purpose: to shove advertisements and subscriptions onto you. My mother's new Windows 11 laptop's out-of-the-box configuration had me clicking through half a dozen things attempting to manipulate me or her into spending more money. There are (I can only assume paid-placement) news and adfotainment in the start menu! Repeat pop-up reminders from Lenovo to subscribe to their protection package. Emotionally-manipulative reminders to subscribe to virus protection services. To Microsoft Office. Etc. etc. It's been the same thing in the mobile market, where the move to "apps" means you are running their software on your device all the time, so they can optimally surveil you, and target the advertisements and behaviourally-modifying nudges. Quite a few messaging services now actively mess with delivery of notifications, spacing them out, delaying them, according to research that shows what maximizes engagement. I saw the trend 20 years ago and switched to free software around that time -- I liked Linux anyway, but it was partly on principle. Still, the new laptop was eye-opening. The degree of intrusion, the degree to which even desktop computers have turned into user-hostile advertising terminals serving the purposes of their manufacturer, rather than a computer for the user to accomplish their work, is quite shocking. Everything networked is becoming like that - twisting the user's hardware, turning it into nothing more than a terminal, an extension of the corporation, serving their interests at all times. Even smart TVs now have ads built-in to their menus and such.
  - mrguyorama 3y ago
    >to shove advertisements and subscriptions onto you. There is no other end state in capitalism. If you want tools and products that serve you instead of an owner, you must do it outside capitalism like with truly open source stuff.
dontreact 3y ago
Is it possible for them to implement this API in such a way that it will fail 5% of the time or so, making it impossible for websites to deny individuals based on failing attestation? https://github.com/RupertBenWiser/Web-Environment-Integrity/issues/16 https://github.com/RupertBenWiser/Web-Environment-Integrity/...
- FireBeyond 3y ago
  ... until Google decides to dial that down to zero for "experience", or Hulu / Netflix makes you disable/whitelist/whatever to access their site.
  - dontreact 3y ago
    But as mentioned above, isn't doing so against Google's own self interest? It seems like the project is explicitly stating their goal isn't to allow for websites to do this, and they are implementing it in a manner consistent with that. One thing about your comment above: Hulu can't start implementing attestation until Google turns the knob to 0 because they can't start randomly dropping 5% of Chrome users. So in your comment above it should be "and" not "or". If I understand correctly Hulu cannot act unilaterally with the currently planned implementation of this. If let's say they did turn the knob for Chrome, wouldn't it take a while for websites to start implementing this? For me not knowing as much about this it feels like this is a step in an ambiguous direction which could be good or bad still. But since it's Google everyone is thinking ahead in the causal chain. Can you help me understand why this is such a big and clearly bad step against the open web? Thank you!
    - WorldMaker 3y ago
      I think Hulu is a great example. Hulu has DRM issues in Firefox and their DRM just fails with unknown errors on about ~15% of content they host (anecdotally, of course, I have no specific data). There's no way for me to tell if a specific episode of a show will fail or not, some succeed, others don't. I at least find no pattern for this. From this perspective, they are essentially randomly breaking 100% of Firefox users some seemingly random percentage of the time. They have "good" business reasons to require this DRM and whatever this random broken user percentage is, I'm sure it meets their bottom-line criteria as a business. "95%" uptime for Chrome users is only "one-9", but it's still got that one 9. That's an acceptable SLA to many businesses. A business might easily decide attestation is worth that "uptime risk" because it sells more ads or makes the DRM vendors happier (and thus the content owners are happier) or any other number of "good" business reasons.
bilekas 3y ago
> The current players can whitelist/attest their own clients while categorizing every other scraping clients as bots. I hadn't really considered this. In a roundabout way, is there a process for this to be rejected on grounds of "fair use" limitations?
ivanstojic 3y ago
How would this work against scrapers that are based on driven anpproved browser instances, eg. something like Selenium?
- therein 3y ago
  The browser instance knows it is being driven by an automation agent. If you so wanted, you can actually comment out the code that does that in the browser's code but since this new setup will enable the page to check if you compiled your own browser, they'll be able to incorporate the "isUnderAutomation" flag under the attestation data and that's sealed because you can't build your own browser and have it attest.
derefr 3y ago
> The current players can whitelist/attest their own clients while categorizing every other scraping clients as bots. Can't they already do this by having scrapers send plain-old client certificates? Or even just a request header that contains an HMAC of the URL with a shared secret? Actually, taking a step further back: why does anyone need to scrape their own properties? They can make up an arbitrary backchannel to access that data — just like the one Google uses to populate YouTube results into SERPs. No need to provide a usefully-scrapeable website at all.

by devsda 3y ago

justcool393 3y ago

> If not for other reasons, I can't see how Google a search company can be allowed to push something that can kill competition using its market dominance in other areas like browsers. Because antitrust has been dead for a while. Chrome is a tool to drive people to Google and Google ads and nothing more. I will say, I did appreciate Microsoft having a browser engine with IE and Edge, even if the former was notoriously a pain, it gave competition in the space. Unfortunately, that's not the case anymore and everything is either Chrome (Blink), Firefox (Gecko), or Safari (WebKit). And it's pretty clear what Chrome has done once that have amassed a dominant market share. I'm sure there are Googlers who think they're legitimately making the web a safer place, but I think the real reason is pretty clear if you take a birds eye view.

deleted 3y ago
[deleted]
retrac 3y ago
Unfortunately it seems that nearly all software platforms with a dominant market share, end up degenerating to serve only one purpose: to shove advertisements and subscriptions onto you. My mother's new Windows 11 laptop's out-of-the-box configuration had me clicking through half a dozen things attempting to manipulate me or her into spending more money. There are (I can only assume paid-placement) news and adfotainment in the start menu! Repeat pop-up reminders from Lenovo to subscribe to their protection package. Emotionally-manipulative reminders to subscribe to virus protection services. To Microsoft Office. Etc. etc. It's been the same thing in the mobile market, where the move to "apps" means you are running their software on your device all the time, so they can optimally surveil you, and target the advertisements and behaviourally-modifying nudges. Quite a few messaging services now actively mess with delivery of notifications, spacing them out, delaying them, according to research that shows what maximizes engagement. I saw the trend 20 years ago and switched to free software around that time -- I liked Linux anyway, but it was partly on principle. Still, the new laptop was eye-opening. The degree of intrusion, the degree to which even desktop computers have turned into user-hostile advertising terminals serving the purposes of their manufacturer, rather than a computer for the user to accomplish their work, is quite shocking. Everything networked is becoming like that - twisting the user's hardware, turning it into nothing more than a terminal, an extension of the corporation, serving their interests at all times. Even smart TVs now have ads built-in to their menus and such.
- mrguyorama 3y ago
  >to shove advertisements and subscriptions onto you. There is no other end state in capitalism. If you want tools and products that serve you instead of an owner, you must do it outside capitalism like with truly open source stuff.

dontreact 3y ago

Is it possible for them to implement this API in such a way that it will fail 5% of the time or so, making it impossible for websites to deny individuals based on failing attestation? https://github.com/RupertBenWiser/Web-Environment-Integrity/issues/16 https://github.com/RupertBenWiser/Web-Environment-Integrity/...

FireBeyond 3y ago
... until Google decides to dial that down to zero for "experience", or Hulu / Netflix makes you disable/whitelist/whatever to access their site.
- dontreact 3y ago
  But as mentioned above, isn't doing so against Google's own self interest? It seems like the project is explicitly stating their goal isn't to allow for websites to do this, and they are implementing it in a manner consistent with that. One thing about your comment above: Hulu can't start implementing attestation until Google turns the knob to 0 because they can't start randomly dropping 5% of Chrome users. So in your comment above it should be "and" not "or". If I understand correctly Hulu cannot act unilaterally with the currently planned implementation of this. If let's say they did turn the knob for Chrome, wouldn't it take a while for websites to start implementing this? For me not knowing as much about this it feels like this is a step in an ambiguous direction which could be good or bad still. But since it's Google everyone is thinking ahead in the causal chain. Can you help me understand why this is such a big and clearly bad step against the open web? Thank you!
  - WorldMaker 3y ago
    I think Hulu is a great example. Hulu has DRM issues in Firefox and their DRM just fails with unknown errors on about ~15% of content they host (anecdotally, of course, I have no specific data). There's no way for me to tell if a specific episode of a show will fail or not, some succeed, others don't. I at least find no pattern for this. From this perspective, they are essentially randomly breaking 100% of Firefox users some seemingly random percentage of the time. They have "good" business reasons to require this DRM and whatever this random broken user percentage is, I'm sure it meets their bottom-line criteria as a business. "95%" uptime for Chrome users is only "one-9", but it's still got that one 9. That's an acceptable SLA to many businesses. A business might easily decide attestation is worth that "uptime risk" because it sells more ads or makes the DRM vendors happier (and thus the content owners are happier) or any other number of "good" business reasons.

bilekas 3y ago

> The current players can whitelist/attest their own clients while categorizing every other scraping clients as bots. I hadn't really considered this. In a roundabout way, is there a process for this to be rejected on grounds of "fair use" limitations?

ivanstojic 3y ago

How would this work against scrapers that are based on driven anpproved browser instances, eg. something like Selenium?

therein 3y ago
The browser instance knows it is being driven by an automation agent. If you so wanted, you can actually comment out the code that does that in the browser's code but since this new setup will enable the page to check if you compiled your own browser, they'll be able to incorporate the "isUnderAutomation" flag under the attestation data and that's sealed because you can't build your own browser and have it attest.

derefr 3y ago

> The current players can whitelist/attest their own clients while categorizing every other scraping clients as bots. Can't they already do this by having scrapers send plain-old client certificates? Or even just a request header that contains an HMAC of the URL with a shared secret? Actually, taking a step further back: why does anyone need to scrape their own properties? They can make up an arbitrary backchannel to access that data — just like the one Google uses to populate YouTube results into SERPs. No need to provide a usefully-scrapeable website at all.