19 ms·

Google vs. the Open Web

544 points by ColinWright 3y ago

troupo 3y ago
The link to Yoav Weiss's blog is great. --- start quote --- So, you don't like a web platform proposal ...you may feel that your insights and experience can be valuable to help steer the platform from making what you're sure is a huge mistake. That's great!! Getting involved in web platform discussions is essential to ensure it's built for and by everyone. ... In cases where controversial browser proposals (or lack of adoption for features folks want, which is a related, but different, subject), it's not uncommon to see issues with dozens or even hundreds of comments from presumably well-intentioned folks, trying to influence the team working on the feature to change their minds. In the many years I've been working on the web platform, I've yet to see this work. Not even once. --- end quote --- "We do so love for everyone to join the discussion. It also never influences our decisions, not once"
- theK 3y ago
  There's two aspects to that actually. 1. Often the feedback goes completely to the wrong address. You won't stop Google from doing google things. 2. Most often the depth level at which the discussions on web standard are made will alienate most people, so instead of participating in "standards making" they turn somewhere else (1.). The web is awesome and it got awesome because for the first 15 years of its existence it was actually very straight forward to run a web entity. But success brought ever growing companies and ever more complex interests. The discussions also vary a lot nowadays. There are still things being done to make the web more approachable but at the same time we see stuff like "Web Environment Integrity", DRM etc. The problem is that a process that requires the public to be vigilant will eventually fail if the public cannot appoint people to be vigilant full time for them.
  - troupo 3y ago
    > Most often the depth level at which the discussions on web standard are made will alienate most people, so instead of participating in "standards making" they turn somewhere else It also takes a lot of time. You have to read quite a few proposals, and there are literally hundreds of them, you have to participate in discussions in the GitHub issues, on the w3c mailing list, and in multiple face-to-face discussions. Even the most technical people find this daunting because they are not paid for this (unlike the people making and promoting the specs). So even the technical people often come into an issue, voice their concerns briefly (or not-so-briefly) and are summarily dismissed. I've seen Google engineers misrepresent and ignore any input from engineers working on Firefox and Safari, and just push their specs forward. So what chance does an outsider have? It's a mess. Granted, it's a better mess because so many discussions are happening in the open unlike 10-15 years ago, but it's still a mess.
    - theK 3y ago
      Agreed. As I say in the original comment > process will ... eventually fail if the public cannot appoint people to be vigilant full time for them. And I was exactly thinking of paid full time work.
- ajross 3y ago
  Clearly the implication is that rushing to join a professional discussion just to yell about some or another controversial proposal you read about on HN is not going to work to sway the stakeholders. If you want influence, you need to cultivate it over time by building trust in the community you want to influence. That's hardly controversial. In particular, taking a fairly dry proposal like WEI, which is intended as a anti-bot/anti-cheat framework for web content, and spinning it with a shitpost title like "Google vs. the Open Web" is really not going to ingratiate you with the people who think hard about very difficult problems every day. Is it a good proposal? Honestly I don't know. But the problems it's trying to address are real, so I'm inclined to give the benefit of the doubt to the people trying to solve them in good faith over the shitposters.
  - lambic 3y ago
    Your definition of shitposting is... odd.
    - ajross 3y ago
      Characterizing a fairly reasonable position like "we should have a way prevent bots at the client level" as an attack on "The Open Web" is pretty much the definition of a shitpost, no? It's a terrible strawman and it pollutes the discourse. Do you really not agree that people might want the former and not the latter? You genuinely think that the standards folks are being driven by a conspiracy and not what they say they want? There are ways to argue against WEI that don't involve the existence of enemies you have to fight. Maybe you could try them?
      - lambic 3y ago
        1. (noun) any content on the internet whose humor derives from its surreal nature and/or its lack of clear context. Differs from a meme: whereas a meme's humor comes from its repeatability, a shitpost is funny simply because it isn't a predictable repetition of an existing form. Shitposts can become memes, but memes cannot become shitposts. 2. (verb) to create such a post
- qjx 3y ago
  For a personal blog it has quite a lot of PR speak
- ticviking 3y ago
  The only solution to this kind of thing is to actually roll up our sleeves and make alternatives. Most people don't have either the skill or time to do that. So we bikeshed instead.
sakex 3y ago
Soon, websites will require kernel access to make sure you don't have cheats installed. (Sarcasm, obviously)
- hotstickyballs 3y ago
  Not that extreme but some banking apps on android did check for root at some point and refused to run so there may be precedent
  - theK 3y ago
    In the 2000s it was funny how corporates just failed to understand how client server models worked. Nowadays it is just sad and a reason to move more and more towards crypto for the day to day banking stuff...
    - theK 3y ago
      Oh right! I said crypto on hacker news! what was I thinking?! :-D
      - nobody9999 3y ago
        >Oh right! I said crypto on hacker news! what was I thinking?! :-D Post that comment again when crypto accounts are FDIC[0] (or whatever scheme, if any, is used where you live) insured. I'm sure you'll get a different response. [0] https://www.fdic.gov/ https://www.fdic.gov/
        theK 3y ago
        You only need FDIC in systems where you don't own the coins. Cryptocurrencies actually obviate a lot of added fat like this ;-)
  - Alifatisk 3y ago
    Remember when Ebay did portscans on their clients to see if it was a bot?
dschuetz 3y ago
The bright side is, when Google really pushes through with this WEI nonsense, it will not only break the Web, it will also create some kind of premium Web run by Google, analogous to gated communities. And then the worldwide Internet ad market bubble will finally burst.
- christophilus 3y ago
  GoogAOL Web 3.0
  - Izkata 3y ago
    ...send it to the gooGAOL
- teddyh 3y ago
  Called it: <https://news.ycombinator.com/item?id=31835121 https://news.ycombinator.com/item?id=31835121> <https://news.ycombinator.com/item?id=33210846 https://news.ycombinator.com/item?id=33210846>
- lost_tourist 3y ago
  eh I've heard this for 20 years...
andy99 3y ago
How does WEI work with non-browsers, like curl or python requests? I was wondering if there is some motive here to monopolize web scraping (especially with respect to harvesting AI training data)?
- phpnode 3y ago
  it doesn't. Hosts will inevitably start blocking clients which don't exchange an attestation token
  - theK 3y ago
    Yeah but the process to exchange the token will be "open sauce"! </troll>
- AndroTux 3y ago
  I mean that’s part of the point. It’s there to exactly lock out scrapers. Or crawlers, for that matter. What a happy little coincidence.
  - Roark66 3y ago
    This is just silly, there exist frameworks like selenium that allow you to run any browser of choice and emulate actual user behavior(clicks, keystrokes). If they go further the emulation layer will have to be moved higher, above the virtual machine running the browser for example. The truth is, this has nothing to do with scraping, scrapers will find a way. This is to stop the majority of people from using ad block.
    - AndroTux 3y ago
      If I understand this proposal correctly, this is exactly to prevent such things. Yes, of course, it’s to prevent people from using ad block. But a nice side-effect is to block crawlers, or frameworks like selenium as well, so they can „serve ads only to real people“. Of course, people will always find a way to crawl. We already have bot farms that are just remote controlled smartphones lined up somewhere. But it makes it harder for everyone who isn’t Google to compete with Google.
Alifatisk 3y ago
So, how can this be bypassed in theory? Any ideas? Brainstorming is allowed. EDIT: Saw a few mention two solutions to disable the automatic verification on iOS & macOS. https://blog.cloudflare.com/how-to-enable-private-access-tokens-in-ios-16-and-stop-seeing-captchas https://blog.cloudflare.com/how-to-enable-private-access-tok... https://support.apple.com/en-us/HT213449 https://support.apple.com/en-us/HT213449
- dschuetz 3y ago
  Build a new Web w/o Google.
  - EGreg 3y ago
    We have been doing it! Join us! It isn’t perfect but we are ahead of most others (Mastodon, Matrix). We have spent TWELVE YEARS building the free, permissionless open source platform for anyone to assemble and host their own community software with all the features of Facebook/Twitter/TikTok for their own community: https://github.com/Qbix/Platform https://github.com/Qbix/Platform We are about to roll out version 2.0 — I have never done this before but I would like to invite whoever wants to learn about it or build on it, to a Zoom webinar where I will demo anything and answer any questions. Starting in Q3 this year all the webinars will take place on our own platform — no Calendly, no Zoom, no Google, just the free open Web. Anyway, sign up here if you want. Will do it every Sunday throughout August: https://calendly.com/qbix/qbix-2-0-platform-demo https://calendly.com/qbix/qbix-2-0-platform-demo Whether you’re a developer, a businessperson, or just want to learn about the latest technologies moving the Free Open Source Web forward, this platform can help empower you to build and engage a community around yourself and your projects.
    - Dah00n 3y ago
      Interesting, if it isn't US based and if it isn't using US politics as a guideline (like banning russian patches). I have lost a big part of my former trust and want for writing OSS this last few months and one thing I have learned is that if those two can't be answered with a resounding no it is a project I won't ever contemplate even though I'm neither American nor Russian.
tb_technical 3y ago
Could Google be taken to task by the FTC on this issue?
- JimtheCoder 3y ago
  Well, Lina Khans performance has been stellar thus far, so I'm sure they'll get right on this...
  - Dah00n 3y ago
    Who has done better?
    - AlexandrB 3y ago
      No kidding. Previous FTCs sat on their hands and allowed Meta (nee Facebook) to acquire their up-and-coming competitors with no pushback.
    - OfferFun6595 3y ago
      I'm keen on this as well
supriyo-biswas 3y ago
I mentioned this in the other WEI thread and I’ll do it here again: Instead of simply flailing our collective arms around complaining about an evil corporation, has anyone written to the respective competition authorities (such as the FTC in the US or CCI in India) about the potential anticompetitive effects of this proposal?
- rapsey 3y ago
  FTC right now has awful leadership. They only care about blocking mergers and scoring political points.
  - freeone3000 3y ago
    Isn’t blocking mergers one of the founding goals of the FTC?
    - scrum-treats 3y ago
      Protecting consumers is the goal. If that means blocking mergers to protect consumers, then so be it.
  - supriyo-biswas 3y ago
    That is a defeatist attitude; writing to them couldn't be worth less than shouting into the abyss with our comments on Github issues and on HN, etc.
    - j45 3y ago
      One thing that I’ve heard before is if one person complains a politician knows there’s 9 more.
  - maxsilver 3y ago
    It's literally the FTC's job to block most mergers (they are required by law to block bad mergers, see https://www.ftc.gov/advice-guidance/competition-guidance/guide-antitrust-laws/mergers https://www.ftc.gov/advice-guidance/competition-guidance/gui... for details) Their leadership isn't "awful", their leaders are finally doing their job, for the first time in decades.
lancesells 3y ago
I'm of the opinion that Google is no longer "Organizing the world's information" but "Stealing the world's information". Their last keynote proved that with all of the AI products. Regardless of this proposal (which is awful) they are a data vampire using all your info for themselves without permission or compensation.
- smoldesu 3y ago
  It's not stealing if the content is lawfully acquired: https://en.wikipedia.org/wiki/Authors_Guild%2C_Inc._v._Google%2C_Inc https://en.wikipedia.org/wiki/Authors_Guild%2C_Inc._v._Googl....
- runiq 3y ago
  Y'all ought to read Rainbow's End, where a Google-alike is trying to OCR all written information to make it accessible online… while shredding the originals in the process. That book was prescient on so many levels. Same with Accelerando.
  - matkoniecz 3y ago
    > while shredding the originals in the process In-universe reason for that made no sense whatsoever and was nonsense. It was not even an evil organization (this would make sense) but OCR method as described was absurd and would not work.
CrzyLngPwd 3y ago
Why don't we just skip to the part where google runs and owns everything and everyone, and we all have to give them 50% of our harvests.
- AlexandrB 3y ago
  I wonder what Google's version of "prima nocta" will look like.
  - CrzyLngPwd 3y ago
    Not limited to females, obvs :-p
  - Slava_Propanei 3y ago
    [dead]
lrem 3y ago
> In general, bots are pretty easy to exclude. They usually advertise themselves by a user agent string. Yes, that can be faked – but it seems highly unlikely that bots using faked user agents create such a large number of impressions that Google has to use this route against them. Is this serious?
- klempner 3y ago
  Yeah, this guy lost me right there. Speaking as someone who works at Google, but not on anything at all related to ads, browsers, or ad spam detection, I only wish that the attackers who (try to) make a living out of scamming Google and its advertisers out of money were as incompetent as the author of this article appears to be.
- jsnell 3y ago
  The author is embarrassingly uninformed on this. There are two kinds of bots. There's legits ones that the site owners will generally find to provide a positive tradeoff. These bots identify themselves by the user-agent, the requests come from a predictable set of IPs, and the they obey the robots.txt. Think most crawlers for search engines (though not Brave's), bots that handle link previews for apps like WhatsApp, even RSS readers! Then there's the abusive ones. These are usually hitting resources that are expensive and contain valuable information. The will not obey robots.txt. They'll run residential IP botnets to avoid IP blocks. They'll make their bot as similar to legit traffic as possible, the user-agent is literally the first thing they'd look at changing. They'll hire mechanical turks to create fake accounts to look like signed in users. Now, it's pretty obvious why the author's methodology for supporting the statement is so silly. First, it was circular. They identified bots by user-agent, and then declared that since there were bots that had a distinguishing user-agent, the other traffic can't have been bots. The other is that they looked at the logs of a server that doesn't contain any data that somebody would be scraping maliciously. Ocean's 11 will do a heist of a casino, not a corner store. Likewise the professional bot operations are scraping valuable information from people trying to actively defend against it, not your blog.
  - leeoniya 3y ago
Communitivity 3y ago
Another year closer to a Shadowrun future, minus the magic, where the most powerful corporations run everything, are more powerful than most nation states, and where your only allowed role in life is a consumer/corpse-servant for life (unless you want to risk the harsh penalties of being illegal and running the shadows). I think we will get to a PostCapitalist future. The decisions we make in the next 7 years will likely determine whether the probable future is dystopian like Shadowrun, or utopian like Paul Mason (see his book "Postcapitalism: a Guide to Our Future"). Personally, I prefer Mason's, with his goals of: - Rapidly reduce carbon emissions to stay below 2 °C warming by 2050 (edit: We've lost this battle, see the current 6-sigma sea ice event and recent AMOC reports - maybe we can hold it to 3 °C). - Stabilise and socialise the global finance system. - Prioritise information-rich technologies to deliver material prosperity and solve social challenges such as ill health and welfare dependency. - Gear technology towards minimising necessary work, until work becomes voluntary and economic management can focus on energy and resources rather than capital and labour. That will not be if we do not bring to heel the FAANG companies now, and prevent things like Apple's Private Access Token, Google's WEI, etc. from taking root (yanking them out of the ground where already present).
- Dah00n 3y ago
  Unless the US disappears it seems pretty clear which of those two choices we will end up with, unless of course there are other choices.
- kmeisthax 3y ago
  The sort of Shadowrun/Snow Crash/ancap future you're talking about is what Cory Doctorow is calling technofeudalism[0]: one in which the primary driver of economic activity reverts to passive income scams[1] instead of active economic activity. [0] https://pluralistic.net/2023/07/24/rent-to-pwn/ https://pluralistic.net/2023/07/24/rent-to-pwn/ [1] Economists call these "rents", even though they're more general than just rent paid to borrow some real property
- teddyh 3y ago
  > Another year closer to a Shadowrun future, minus the magic The usual term for that is “cyberpunk dystopia”.
Daril 3y ago
This is a clear sign of Google's weakness. They are losing their monopoly and are desperately trying to hold on to the net. In the last few weeks, they have announced that they will try to block navigation if you have an ad blocker installed (for example, when watching a video on Youtube). Take a look at Fuchsia for another example ... they are losing the control on Android, so they started this new project ... it is another sign. My recipe: AdGuard Home, Brave browser (phone, tablet, desktop), Bromite (phone), Firefox (desktop) + uBlock origin plugin ... and FreeTube on desktop. Just using Brave on the phone is enough to kill all ads and trackers. In the open source community, there will always be someone smarter than they think who will find a way around their gates... Few days ago Kevin Mitnick passed away, sadly, but there will be always another Kevin Mitnick ... Google will lose all respect from the community and will collapse sooner or later.
- andsoitis 3y ago
  > They are losing their monopoly and are desperately trying to hold on to the net > they are losing the control on Android What do you mean by this and what does Android have to do with trying to hold on to the net?
  - gkbrk 3y ago
    A huge number of people only use the internet with their phones, and Google is doing their best to tie the entire system to Google services and the Chrome browser.
    - Daril 3y ago
      Exactly. And why the phones are sold with preinstalled with all the Google software and services and it is difficult to remove them if not via adb cli ? If they are the best in the World as someone else has stated, the people will flock to install them on a "vanilla" phone, right ? All your SMS, contacts, emails, location positions, photos ... One time my mother was at a funeral at a cemetry, nearby is a cafè. 20 minutes later she got a notification from Google : "How was your experience at the cafè ?" What have this to do with their own products, their free tier levels, the freedom to not watch Youtube or use Gmail ?
gjsman-1000 3y ago
I'm sorry, but using the term "generative system" and complaining that this undermines the internet founded on "generative systems" is perhaps the least impressive way to get anyone to care about the open web. Using buzzwords from some random paper just overwhelms people and doesn't convince them to care. Tell my uncle, or my aunt, that "Google wants to undermine the internet of generative systems!" Whatever. Tell them "Google wants websites to be able to block any devices you might have modified, in any way, that the website owner doesn't like" and you'll get a much stronger reaction.
- kmeisthax 3y ago
  Even that won't get your uncle or aunt to speak up. "Google wants to block any devices you repaired yourself" might get some traction.
cavisne 3y ago
“They usually advertise themselves by a user agent string” my understanding is this is very much not true, but all that info comes from cloudflare which obviously has an incentive. Are there stats on this?
- colonwqbang 3y ago
  No that part of the article was very silly, almost dishonest as I imagine he actually knows better. > If I look at my own webserver logs, it’s very clear which are bot requests Nobody ever lies on the internet?
pabs3 3y ago
I wonder if any web servers or web apps have started to block Chrome users yet.
protocolture 3y ago
This is kind of overblown isnt it? I remember sites doing all sorts of hacks to identify and shut down IE back in the day. "Works best in Chrome/Firefox". "The proposal calls for at least the following information in the signed attestation: The attester's identity, for example, "Google Play". A verdict saying whether the attester considers the device trustworthy. " So a user agent string and a weak attestation? This seems an overcomplex nothingburger.
- nobody9999 3y ago
  And if the "attester" decides that IceWeasel on Ubuntu (or Firefox with uBlock/uMatrix/NoScript) isn't "trustworthy," but (unmodified) Chrome is "trustworthy," you've just created vendor lock-in. That's not a "nothingburger" IMHO.
  - protocolture 3y ago
    The same proposal suggests that users who fail the attestation still access the content. Which is apparently how the Apple version of this same protocol already works.
- helen___keller 3y ago
  It’s a signed attestation. A user agent can be spoofed, this attestation needs to be signed cryptographically with a trusted key, for example a hardware key shipped in your device by an approved vendor. Think Apples Secure Enclave. The goal is a verified stack - the hardware key proves you have approved hardware. The approved hardware proves you don’t have a tampered OS. The untampered OS proves you have approved binaries. The approved binaries disallow certain actions that users want such as blocking ads or downloading YouTube videos.
- allisdust 3y ago
  What part of attestation don't you understand? If linked with a OS level signing with keys stored on TPM, it's game over for private browsing. The only thing worse than companies proposing such measures are the useful idiots downplaying the impact. If someone disagrees, pray tell us muddle brains how to bypass this on a proprietary OS with locked boot and tpm stored keys.
tlogan 3y ago
This is how the situation will unfold... - The WEI check will be designed with a level of simplicity that tech-savvy individuals or hackers can easily bypass. Criticisms or objections will be quieted with comments like, "You just need to initiate the browser using these 50 different settings and you're good." - On the other hand, the WEI check will be intricate enough that an average user won't be able to circumvent it, resulting in them being obligated to view ads. In this way, it's a win-win situation: the hackers maintain their access to an "open" web, while the vast majority (99%) of the population will navigate through a "Google" web.
- kmeisthax 3y ago
  WEI is a proxy to Play Protect which is already a pain in the ass to circumvent for techies.
- pmlnr 3y ago
  Try using https://microg.org/ https://microg.org/ to replace Google Play Services - it's not your average tech-savvy level.
ocdtrekkie 3y ago
I'm still really irritated by all the people at Google who claimed "Team Web" like they were the good guys, while working on things like AMP designed to proprietize the web. What we really need is to get the W3C and IETF to straight up throw out vendors who repeatedly push user hostile proposals. Do not let anyone with employed by Google contribute a web standard. Period. And reevaluate the ones already accepted while we're at it.
darklycan51 3y ago
I actively despise google and avoid any product they make (except youtube, got no choice there) for this reason. It goes back all the way 2010, I remember opening up chrome to try it, right clicking on a youtube channel background to attempt to download like I could on firefox just fine, and it not having the option, why would you go out of your way to restrict a user easily being able to right click and download? well, because you believe you own the web. Never used Chrome and never will, if you use Chrome you are actively making it worse for yourself in the future once they implement enough bad policies that it becomes near impossible for almost anyone or anyone to bypass their restrictions.
Dah00n 3y ago
Smells of vendor lock-in. Someone in the EU should report this. It is quite easy: https://europa.eu/youreurope/business/selling-in-eu/competition-between-businesses/anti-competitive-behaviour/index_en.htm https://europa.eu/youreurope/business/selling-in-eu/competit...
AlexandrB 3y ago
Makes me sad to think how locked-down modern computing is becoming. Between app stores, DRM, TPM, and proposals like WEI future generations of hackers will have a very different experience of what you can and can't do with a computer than I did.
- morkalork 3y ago
  The comment about it killing scraping makes me sad. Figuring out a website's api and collecting your own dataset using python+scrapy for personal ML projects is a wonderful learning exercise that I recommend to everyone. A world of only approved datasets from Kaggle etc. is not the same.
- skydhash 3y ago
  There are still single board computers and Linux to play with.
xbmcuser 3y ago
Its sad for me to see Google being herded in this direction over the last few years. Google was one of the main push behind an open web thorough 2000s-2010s as they wanted data for search and when everything was open they had access to everything. But as new web 2.0 companies came about like facebook that started siloing the internet it started changing things. I have been anti facebook for this reason alone not its data mining etc but because it was the reason the web started to change where instead of websites a lot of companies started building facebook pages with the data not being available unless you are logged in to facebook.
- kccqzy 3y ago
  Yes. Wasn't Google also one the main forces behind PWAs? Google also introduced so many new web APIs (even WebUSB!) in an attempt to make web apps competitive with native apps and their native APIs. To resolve this conundrum, Google as a whole cannot be said to be "for" or "against" the open web. Instead, Google's infamous internal infighting means that you can only say some parts of Google are for the open web, others are against, and sometimes one has an upper hand.
cma 3y ago
Break up Google
pphysch 3y ago
Frankly, it's long past time we had "anti-cheat" available for the web. People vastly underestimate how bad "web cheaters" are for society. They drown search engines and social networks with spam that advances interests for a tiny minority. Places like Reddit are flooded with astroturfed misinformation. They are not as obvious as cheaters in a video game, but far more consequential. There are obviously a lot of details to work out, but pretending that this is just a power grab by Apple, Google et al. instead of an attempt to address an extremely serious problem (that HN indirectly complains about all the time, e.g. "web 2.0 sucks", "why are search results so bad now"), is just naïve.
krono 3y ago
Nearly ever single product, service, and idea that Google brought into this world beyond - perhaps - their early days search engine, was created in response to the rise of some problem of Google's own single-handed making. Inadvertently at first, they were amongst the first to ever even run into some of the complexities of providing internet services at scale after all. But then the rot took hold and Google Chrome was created.
charcircuit 3y ago
>but it seems highly unlikely that bots using faked user agents create such a large number of impressions that Google has to use this route against them Clickfaud makes the frauders money. There is a financial incentive in fooling Google's bot detection. There is a lot of bets using a faked user agent. >Websites can refuse service unless you install their proprietary data collection agent. Websites can refuse service if you use the wrong browser Websites can already do this.
multicast 3y ago
How can a bot create fake impressions? When a bot (or just a simple program) makes a http request he fetches the raw html code only. AFAIK if you don't actually render the html code in a browser or requesting all the contents afterwards again with http requests (like GET ad.jpg, GET logo.png etc.), no google ad server should be hit. Now you could argue that bots could inflate the popularity of a website and therefore the cost to run ads on it. But I guess websites that show ads have most likely google analytics running, one of the only ways Google can actually calculate the popularity (besides Google Search and maybe Chrome history). So it should be no problem for Google to exclude bots from the popularity calculation by analyzing traffic. Maybe I am just missing something, I am also no ad expert at all.
- anchovy_ 3y ago
  Don't nail me down on this but I think since nowadays' websites are often dynamic, you most likely have to employ headless browsers in order to do whatever it is you want to do. This should then result in fake impressions.
  - multicast 3y ago
    Good point. Haven't thought about that.
- jsnell 3y ago
  It's not about bots creating fake ad impressions by accident. It's people writing bots whose purpose is to fake ad impressions and clicks. They'll then run it on their own website that's running ads, with the goal of being paid by the ad network for this fake traffic.
  - multicast 3y ago
    But isn't this a win situation for Google to a certain extent? Since it uses up the budget of the advertiser much faster. And the accuracy of filtering new revenue coming from ads as a company is already fairly limited in general. But maybe there are multiple reasons that Google really only wants to serve real humans to the ads of its clients.
ryu2k2 3y ago
I don't want to play devil's advocate, but when I consider the history of personal computing it would almost surprise me if this didn't get through. From locked-down smartphones having become the norm to windows having turned into a carrier for advertisements, this just seems like the next step in some inevitable evolution of IT. Maybe news agencies aren't much interested because this is still only an early proposal, but considering its implications I find it striking how little it seems to be talked about on the web (outside of hackernews). Rossmann seems to be the only one with a video on the topic up on Youtube. There's only a handful of Twitter results for "web environment integrity api" with next to no replies to them. When I look the keystring up on reddit the only result with a noteworthy amount of debate is not related to WEI at all. Social media is probably just on its last legs, but it doesn't seem like too many people that care are left to fight for what the web, or even general computing, used to be.
keybits 3y ago
Related to WEI, Apple already has something similar live for Safari users, see: "Apple already shipped attestation on the web, and we barely noticed" - https://httptoolkit.com/blog/apple-private-access-tokens-attestation/ https://httptoolkit.com/blog/apple-private-access-tokens-att...
Young-Lord 3y ago
Let's take action now. https://github.com/Young-Lord/fight-for-the-open-web https://github.com/Young-Lord/fight-for-the-open-web

544 points by ColinWright 3y ago

The link to Yoav Weiss's blog is great. --- start quote --- So, you don't like a web platform proposal ...you may feel that your insights and experience can be valuable to help steer the platform from making what you're sure is a huge mistake. That's great!! Getting involved in web platform discussions is essential to ensure it's built for and by everyone. ... In cases where controversial browser proposals (or lack of adoption for features folks want, which is a related, but different, subject), it's not uncommon to see issues with dozens or even hundreds of comments from presumably well-intentioned folks, trying to influence the team working on the feature to change their minds. In the many years I've been working on the web platform, I've yet to see this work. Not even once. --- end quote --- "We do so love for everyone to join the discussion. It also never influences our decisions, not once"

theK 3y ago
There's two aspects to that actually. 1. Often the feedback goes completely to the wrong address. You won't stop Google from doing google things. 2. Most often the depth level at which the discussions on web standard are made will alienate most people, so instead of participating in "standards making" they turn somewhere else (1.). The web is awesome and it got awesome because for the first 15 years of its existence it was actually very straight forward to run a web entity. But success brought ever growing companies and ever more complex interests. The discussions also vary a lot nowadays. There are still things being done to make the web more approachable but at the same time we see stuff like "Web Environment Integrity", DRM etc. The problem is that a process that requires the public to be vigilant will eventually fail if the public cannot appoint people to be vigilant full time for them.
- troupo 3y ago
  > Most often the depth level at which the discussions on web standard are made will alienate most people, so instead of participating in "standards making" they turn somewhere else It also takes a lot of time. You have to read quite a few proposals, and there are literally hundreds of them, you have to participate in discussions in the GitHub issues, on the w3c mailing list, and in multiple face-to-face discussions. Even the most technical people find this daunting because they are not paid for this (unlike the people making and promoting the specs). So even the technical people often come into an issue, voice their concerns briefly (or not-so-briefly) and are summarily dismissed. I've seen Google engineers misrepresent and ignore any input from engineers working on Firefox and Safari, and just push their specs forward. So what chance does an outsider have? It's a mess. Granted, it's a better mess because so many discussions are happening in the open unlike 10-15 years ago, but it's still a mess.
  - theK 3y ago
    Agreed. As I say in the original comment > process will ... eventually fail if the public cannot appoint people to be vigilant full time for them. And I was exactly thinking of paid full time work.
ajross 3y ago
Clearly the implication is that rushing to join a professional discussion just to yell about some or another controversial proposal you read about on HN is not going to work to sway the stakeholders. If you want influence, you need to cultivate it over time by building trust in the community you want to influence. That's hardly controversial. In particular, taking a fairly dry proposal like WEI, which is intended as a anti-bot/anti-cheat framework for web content, and spinning it with a shitpost title like "Google vs. the Open Web" is really not going to ingratiate you with the people who think hard about very difficult problems every day. Is it a good proposal? Honestly I don't know. But the problems it's trying to address are real, so I'm inclined to give the benefit of the doubt to the people trying to solve them in good faith over the shitposters.
- lambic 3y ago
  Your definition of shitposting is... odd.
  - ajross 3y ago
    Characterizing a fairly reasonable position like "we should have a way prevent bots at the client level" as an attack on "The Open Web" is pretty much the definition of a shitpost, no? It's a terrible strawman and it pollutes the discourse. Do you really not agree that people might want the former and not the latter? You genuinely think that the standards folks are being driven by a conspiracy and not what they say they want? There are ways to argue against WEI that don't involve the existence of enemies you have to fight. Maybe you could try them?
    - lambic 3y ago
      1. (noun) any content on the internet whose humor derives from its surreal nature and/or its lack of clear context. Differs from a meme: whereas a meme's humor comes from its repeatability, a shitpost is funny simply because it isn't a predictable repetition of an existing form. Shitposts can become memes, but memes cannot become shitposts. 2. (verb) to create such a post
qjx 3y ago
For a personal blog it has quite a lot of PR speak
ticviking 3y ago
The only solution to this kind of thing is to actually roll up our sleeves and make alternatives. Most people don't have either the skill or time to do that. So we bikeshed instead.

sakex 3y ago

Soon, websites will require kernel access to make sure you don't have cheats installed. (Sarcasm, obviously)

hotstickyballs 3y ago
Not that extreme but some banking apps on android did check for root at some point and refused to run so there may be precedent
- theK 3y ago
  In the 2000s it was funny how corporates just failed to understand how client server models worked. Nowadays it is just sad and a reason to move more and more towards crypto for the day to day banking stuff...
  - theK 3y ago
    Oh right! I said crypto on hacker news! what was I thinking?! :-D
    - nobody9999 3y ago
      >Oh right! I said crypto on hacker news! what was I thinking?! :-D Post that comment again when crypto accounts are FDIC[0] (or whatever scheme, if any, is used where you live) insured. I'm sure you'll get a different response. [0] https://www.fdic.gov/ https://www.fdic.gov/
      - theK 3y ago
        You only need FDIC in systems where you don't own the coins. Cryptocurrencies actually obviate a lot of added fat like this ;-)
- Alifatisk 3y ago
  Remember when Ebay did portscans on their clients to see if it was a bot?

dschuetz 3y ago

The bright side is, when Google really pushes through with this WEI nonsense, it will not only break the Web, it will also create some kind of premium Web run by Google, analogous to gated communities. And then the worldwide Internet ad market bubble will finally burst.

christophilus 3y ago
GoogAOL Web 3.0
- Izkata 3y ago
  ...send it to the gooGAOL
teddyh 3y ago
Called it: <https://news.ycombinator.com/item?id=31835121 https://news.ycombinator.com/item?id=31835121> <https://news.ycombinator.com/item?id=33210846 https://news.ycombinator.com/item?id=33210846>
lost_tourist 3y ago
eh I've heard this for 20 years...

andy99 3y ago

How does WEI work with non-browsers, like curl or python requests? I was wondering if there is some motive here to monopolize web scraping (especially with respect to harvesting AI training data)?

phpnode 3y ago
it doesn't. Hosts will inevitably start blocking clients which don't exchange an attestation token
- theK 3y ago
  Yeah but the process to exchange the token will be "open sauce"! </troll>
AndroTux 3y ago
I mean that’s part of the point. It’s there to exactly lock out scrapers. Or crawlers, for that matter. What a happy little coincidence.
- Roark66 3y ago
  This is just silly, there exist frameworks like selenium that allow you to run any browser of choice and emulate actual user behavior(clicks, keystrokes). If they go further the emulation layer will have to be moved higher, above the virtual machine running the browser for example. The truth is, this has nothing to do with scraping, scrapers will find a way. This is to stop the majority of people from using ad block.
  - AndroTux 3y ago
    If I understand this proposal correctly, this is exactly to prevent such things. Yes, of course, it’s to prevent people from using ad block. But a nice side-effect is to block crawlers, or frameworks like selenium as well, so they can „serve ads only to real people“. Of course, people will always find a way to crawl. We already have bot farms that are just remote controlled smartphones lined up somewhere. But it makes it harder for everyone who isn’t Google to compete with Google.

Alifatisk 3y ago

So, how can this be bypassed in theory? Any ideas? Brainstorming is allowed. EDIT: Saw a few mention two solutions to disable the automatic verification on iOS & macOS. https://blog.cloudflare.com/how-to-enable-private-access-tokens-in-ios-16-and-stop-seeing-captchas https://blog.cloudflare.com/how-to-enable-private-access-tok... https://support.apple.com/en-us/HT213449 https://support.apple.com/en-us/HT213449

dschuetz 3y ago
Build a new Web w/o Google.
- EGreg 3y ago
  We have been doing it! Join us! It isn’t perfect but we are ahead of most others (Mastodon, Matrix). We have spent TWELVE YEARS building the free, permissionless open source platform for anyone to assemble and host their own community software with all the features of Facebook/Twitter/TikTok for their own community: https://github.com/Qbix/Platform https://github.com/Qbix/Platform We are about to roll out version 2.0 — I have never done this before but I would like to invite whoever wants to learn about it or build on it, to a Zoom webinar where I will demo anything and answer any questions. Starting in Q3 this year all the webinars will take place on our own platform — no Calendly, no Zoom, no Google, just the free open Web. Anyway, sign up here if you want. Will do it every Sunday throughout August: https://calendly.com/qbix/qbix-2-0-platform-demo https://calendly.com/qbix/qbix-2-0-platform-demo Whether you’re a developer, a businessperson, or just want to learn about the latest technologies moving the Free Open Source Web forward, this platform can help empower you to build and engage a community around yourself and your projects.
  - Dah00n 3y ago
    Interesting, if it isn't US based and if it isn't using US politics as a guideline (like banning russian patches). I have lost a big part of my former trust and want for writing OSS this last few months and one thing I have learned is that if those two can't be answered with a resounding no it is a project I won't ever contemplate even though I'm neither American nor Russian.

tb_technical 3y ago

Could Google be taken to task by the FTC on this issue?

JimtheCoder 3y ago
Well, Lina Khans performance has been stellar thus far, so I'm sure they'll get right on this...
- Dah00n 3y ago
  Who has done better?
  - AlexandrB 3y ago
    No kidding. Previous FTCs sat on their hands and allowed Meta (nee Facebook) to acquire their up-and-coming competitors with no pushback.
  - OfferFun6595 3y ago
    I'm keen on this as well

supriyo-biswas 3y ago

I mentioned this in the other WEI thread and I’ll do it here again: Instead of simply flailing our collective arms around complaining about an evil corporation, has anyone written to the respective competition authorities (such as the FTC in the US or CCI in India) about the potential anticompetitive effects of this proposal?

rapsey 3y ago
FTC right now has awful leadership. They only care about blocking mergers and scoring political points.
- freeone3000 3y ago
  Isn’t blocking mergers one of the founding goals of the FTC?
  - scrum-treats 3y ago
    Protecting consumers is the goal. If that means blocking mergers to protect consumers, then so be it.
- supriyo-biswas 3y ago
  That is a defeatist attitude; writing to them couldn't be worth less than shouting into the abyss with our comments on Github issues and on HN, etc.
  - j45 3y ago
    One thing that I’ve heard before is if one person complains a politician knows there’s 9 more.
- maxsilver 3y ago
  It's literally the FTC's job to block most mergers (they are required by law to block bad mergers, see https://www.ftc.gov/advice-guidance/competition-guidance/guide-antitrust-laws/mergers https://www.ftc.gov/advice-guidance/competition-guidance/gui... for details) Their leadership isn't "awful", their leaders are finally doing their job, for the first time in decades.

lancesells 3y ago

I'm of the opinion that Google is no longer "Organizing the world's information" but "Stealing the world's information". Their last keynote proved that with all of the AI products. Regardless of this proposal (which is awful) they are a data vampire using all your info for themselves without permission or compensation.

smoldesu 3y ago
It's not stealing if the content is lawfully acquired: https://en.wikipedia.org/wiki/Authors_Guild%2C_Inc._v._Google%2C_Inc https://en.wikipedia.org/wiki/Authors_Guild%2C_Inc._v._Googl....
runiq 3y ago
Y'all ought to read Rainbow's End, where a Google-alike is trying to OCR all written information to make it accessible online… while shredding the originals in the process. That book was prescient on so many levels. Same with Accelerando.
- matkoniecz 3y ago
  > while shredding the originals in the process In-universe reason for that made no sense whatsoever and was nonsense. It was not even an evil organization (this would make sense) but OCR method as described was absurd and would not work.

CrzyLngPwd 3y ago

Why don't we just skip to the part where google runs and owns everything and everyone, and we all have to give them 50% of our harvests.

AlexandrB 3y ago
I wonder what Google's version of "prima nocta" will look like.
- CrzyLngPwd 3y ago
  Not limited to females, obvs :-p
- Slava_Propanei 3y ago
  [dead]

lrem 3y ago

> In general, bots are pretty easy to exclude. They usually advertise themselves by a user agent string. Yes, that can be faked – but it seems highly unlikely that bots using faked user agents create such a large number of impressions that Google has to use this route against them. Is this serious?

klempner 3y ago
Yeah, this guy lost me right there. Speaking as someone who works at Google, but not on anything at all related to ads, browsers, or ad spam detection, I only wish that the attackers who (try to) make a living out of scamming Google and its advertisers out of money were as incompetent as the author of this article appears to be.
jsnell 3y ago
The author is embarrassingly uninformed on this. There are two kinds of bots. There's legits ones that the site owners will generally find to provide a positive tradeoff. These bots identify themselves by the user-agent, the requests come from a predictable set of IPs, and the they obey the robots.txt. Think most crawlers for search engines (though not Brave's), bots that handle link previews for apps like WhatsApp, even RSS readers! Then there's the abusive ones. These are usually hitting resources that are expensive and contain valuable information. The will not obey robots.txt. They'll run residential IP botnets to avoid IP blocks. They'll make their bot as similar to legit traffic as possible, the user-agent is literally the first thing they'd look at changing. They'll hire mechanical turks to create fake accounts to look like signed in users. Now, it's pretty obvious why the author's methodology for supporting the statement is so silly. First, it was circular. They identified bots by user-agent, and then declared that since there were bots that had a distinguishing user-agent, the other traffic can't have been bots. The other is that they looked at the logs of a server that doesn't contain any data that somebody would be scraping maliciously. Ocean's 11 will do a heist of a casino, not a corner store. Likewise the professional bot operations are scraping valuable information from people trying to actively defend against it, not your blog.
- leeoniya 3y ago

Communitivity 3y ago

Another year closer to a Shadowrun future, minus the magic, where the most powerful corporations run everything, are more powerful than most nation states, and where your only allowed role in life is a consumer/corpse-servant for life (unless you want to risk the harsh penalties of being illegal and running the shadows). I think we will get to a PostCapitalist future. The decisions we make in the next 7 years will likely determine whether the probable future is dystopian like Shadowrun, or utopian like Paul Mason (see his book "Postcapitalism: a Guide to Our Future"). Personally, I prefer Mason's, with his goals of: - Rapidly reduce carbon emissions to stay below 2 °C warming by 2050 (edit: We've lost this battle, see the current 6-sigma sea ice event and recent AMOC reports - maybe we can hold it to 3 °C). - Stabilise and socialise the global finance system. - Prioritise information-rich technologies to deliver material prosperity and solve social challenges such as ill health and welfare dependency. - Gear technology towards minimising necessary work, until work becomes voluntary and economic management can focus on energy and resources rather than capital and labour. That will not be if we do not bring to heel the FAANG companies now, and prevent things like Apple's Private Access Token, Google's WEI, etc. from taking root (yanking them out of the ground where already present).

Dah00n 3y ago
Unless the US disappears it seems pretty clear which of those two choices we will end up with, unless of course there are other choices.
kmeisthax 3y ago
The sort of Shadowrun/Snow Crash/ancap future you're talking about is what Cory Doctorow is calling technofeudalism[0]: one in which the primary driver of economic activity reverts to passive income scams[1] instead of active economic activity. [0] https://pluralistic.net/2023/07/24/rent-to-pwn/ https://pluralistic.net/2023/07/24/rent-to-pwn/ [1] Economists call these "rents", even though they're more general than just rent paid to borrow some real property
teddyh 3y ago
> Another year closer to a Shadowrun future, minus the magic The usual term for that is “cyberpunk dystopia”.

Daril 3y ago

This is a clear sign of Google's weakness. They are losing their monopoly and are desperately trying to hold on to the net. In the last few weeks, they have announced that they will try to block navigation if you have an ad blocker installed (for example, when watching a video on Youtube). Take a look at Fuchsia for another example ... they are losing the control on Android, so they started this new project ... it is another sign. My recipe: AdGuard Home, Brave browser (phone, tablet, desktop), Bromite (phone), Firefox (desktop) + uBlock origin plugin ... and FreeTube on desktop. Just using Brave on the phone is enough to kill all ads and trackers. In the open source community, there will always be someone smarter than they think who will find a way around their gates... Few days ago Kevin Mitnick passed away, sadly, but there will be always another Kevin Mitnick ... Google will lose all respect from the community and will collapse sooner or later.

andsoitis 3y ago
> They are losing their monopoly and are desperately trying to hold on to the net > they are losing the control on Android What do you mean by this and what does Android have to do with trying to hold on to the net?
- gkbrk 3y ago
  A huge number of people only use the internet with their phones, and Google is doing their best to tie the entire system to Google services and the Chrome browser.
  - Daril 3y ago
    Exactly. And why the phones are sold with preinstalled with all the Google software and services and it is difficult to remove them if not via adb cli ? If they are the best in the World as someone else has stated, the people will flock to install them on a "vanilla" phone, right ? All your SMS, contacts, emails, location positions, photos ... One time my mother was at a funeral at a cemetry, nearby is a cafè. 20 minutes later she got a notification from Google : "How was your experience at the cafè ?" What have this to do with their own products, their free tier levels, the freedom to not watch Youtube or use Gmail ?

gjsman-1000 3y ago

I'm sorry, but using the term "generative system" and complaining that this undermines the internet founded on "generative systems" is perhaps the least impressive way to get anyone to care about the open web. Using buzzwords from some random paper just overwhelms people and doesn't convince them to care. Tell my uncle, or my aunt, that "Google wants to undermine the internet of generative systems!" Whatever. Tell them "Google wants websites to be able to block any devices you might have modified, in any way, that the website owner doesn't like" and you'll get a much stronger reaction.

kmeisthax 3y ago
Even that won't get your uncle or aunt to speak up. "Google wants to block any devices you repaired yourself" might get some traction.

cavisne 3y ago

“They usually advertise themselves by a user agent string” my understanding is this is very much not true, but all that info comes from cloudflare which obviously has an incentive. Are there stats on this?

colonwqbang 3y ago
No that part of the article was very silly, almost dishonest as I imagine he actually knows better. > If I look at my own webserver logs, it’s very clear which are bot requests Nobody ever lies on the internet?

pabs3 3y ago

I wonder if any web servers or web apps have started to block Chrome users yet.

protocolture 3y ago

This is kind of overblown isnt it? I remember sites doing all sorts of hacks to identify and shut down IE back in the day. "Works best in Chrome/Firefox". "The proposal calls for at least the following information in the signed attestation: The attester's identity, for example, "Google Play". A verdict saying whether the attester considers the device trustworthy. " So a user agent string and a weak attestation? This seems an overcomplex nothingburger.

nobody9999 3y ago
And if the "attester" decides that IceWeasel on Ubuntu (or Firefox with uBlock/uMatrix/NoScript) isn't "trustworthy," but (unmodified) Chrome is "trustworthy," you've just created vendor lock-in. That's not a "nothingburger" IMHO.
- protocolture 3y ago
  The same proposal suggests that users who fail the attestation still access the content. Which is apparently how the Apple version of this same protocol already works.
helen___keller 3y ago
It’s a signed attestation. A user agent can be spoofed, this attestation needs to be signed cryptographically with a trusted key, for example a hardware key shipped in your device by an approved vendor. Think Apples Secure Enclave. The goal is a verified stack - the hardware key proves you have approved hardware. The approved hardware proves you don’t have a tampered OS. The untampered OS proves you have approved binaries. The approved binaries disallow certain actions that users want such as blocking ads or downloading YouTube videos.
allisdust 3y ago
What part of attestation don't you understand? If linked with a OS level signing with keys stored on TPM, it's game over for private browsing. The only thing worse than companies proposing such measures are the useful idiots downplaying the impact. If someone disagrees, pray tell us muddle brains how to bypass this on a proprietary OS with locked boot and tpm stored keys.

tlogan 3y ago

This is how the situation will unfold... - The WEI check will be designed with a level of simplicity that tech-savvy individuals or hackers can easily bypass. Criticisms or objections will be quieted with comments like, "You just need to initiate the browser using these 50 different settings and you're good." - On the other hand, the WEI check will be intricate enough that an average user won't be able to circumvent it, resulting in them being obligated to view ads. In this way, it's a win-win situation: the hackers maintain their access to an "open" web, while the vast majority (99%) of the population will navigate through a "Google" web.

kmeisthax 3y ago
WEI is a proxy to Play Protect which is already a pain in the ass to circumvent for techies.
pmlnr 3y ago
Try using https://microg.org/ https://microg.org/ to replace Google Play Services - it's not your average tech-savvy level.

ocdtrekkie 3y ago

I'm still really irritated by all the people at Google who claimed "Team Web" like they were the good guys, while working on things like AMP designed to proprietize the web. What we really need is to get the W3C and IETF to straight up throw out vendors who repeatedly push user hostile proposals. Do not let anyone with employed by Google contribute a web standard. Period. And reevaluate the ones already accepted while we're at it.

darklycan51 3y ago

I actively despise google and avoid any product they make (except youtube, got no choice there) for this reason. It goes back all the way 2010, I remember opening up chrome to try it, right clicking on a youtube channel background to attempt to download like I could on firefox just fine, and it not having the option, why would you go out of your way to restrict a user easily being able to right click and download? well, because you believe you own the web. Never used Chrome and never will, if you use Chrome you are actively making it worse for yourself in the future once they implement enough bad policies that it becomes near impossible for almost anyone or anyone to bypass their restrictions.

Dah00n 3y ago

Smells of vendor lock-in. Someone in the EU should report this. It is quite easy: https://europa.eu/youreurope/business/selling-in-eu/competition-between-businesses/anti-competitive-behaviour/index_en.htm https://europa.eu/youreurope/business/selling-in-eu/competit...

AlexandrB 3y ago

Makes me sad to think how locked-down modern computing is becoming. Between app stores, DRM, TPM, and proposals like WEI future generations of hackers will have a very different experience of what you can and can't do with a computer than I did.

morkalork 3y ago
The comment about it killing scraping makes me sad. Figuring out a website's api and collecting your own dataset using python+scrapy for personal ML projects is a wonderful learning exercise that I recommend to everyone. A world of only approved datasets from Kaggle etc. is not the same.
skydhash 3y ago
There are still single board computers and Linux to play with.

xbmcuser 3y ago

Its sad for me to see Google being herded in this direction over the last few years. Google was one of the main push behind an open web thorough 2000s-2010s as they wanted data for search and when everything was open they had access to everything. But as new web 2.0 companies came about like facebook that started siloing the internet it started changing things. I have been anti facebook for this reason alone not its data mining etc but because it was the reason the web started to change where instead of websites a lot of companies started building facebook pages with the data not being available unless you are logged in to facebook.

kccqzy 3y ago
Yes. Wasn't Google also one the main forces behind PWAs? Google also introduced so many new web APIs (even WebUSB!) in an attempt to make web apps competitive with native apps and their native APIs. To resolve this conundrum, Google as a whole cannot be said to be "for" or "against" the open web. Instead, Google's infamous internal infighting means that you can only say some parts of Google are for the open web, others are against, and sometimes one has an upper hand.

cma 3y ago

Break up Google

pphysch 3y ago

Frankly, it's long past time we had "anti-cheat" available for the web. People vastly underestimate how bad "web cheaters" are for society. They drown search engines and social networks with spam that advances interests for a tiny minority. Places like Reddit are flooded with astroturfed misinformation. They are not as obvious as cheaters in a video game, but far more consequential. There are obviously a lot of details to work out, but pretending that this is just a power grab by Apple, Google et al. instead of an attempt to address an extremely serious problem (that HN indirectly complains about all the time, e.g. "web 2.0 sucks", "why are search results so bad now"), is just naïve.

krono 3y ago

Nearly ever single product, service, and idea that Google brought into this world beyond - perhaps - their early days search engine, was created in response to the rise of some problem of Google's own single-handed making. Inadvertently at first, they were amongst the first to ever even run into some of the complexities of providing internet services at scale after all. But then the rot took hold and Google Chrome was created.

charcircuit 3y ago

>but it seems highly unlikely that bots using faked user agents create such a large number of impressions that Google has to use this route against them Clickfaud makes the frauders money. There is a financial incentive in fooling Google's bot detection. There is a lot of bets using a faked user agent. >Websites can refuse service unless you install their proprietary data collection agent. Websites can refuse service if you use the wrong browser Websites can already do this.

multicast 3y ago

How can a bot create fake impressions? When a bot (or just a simple program) makes a http request he fetches the raw html code only. AFAIK if you don't actually render the html code in a browser or requesting all the contents afterwards again with http requests (like GET ad.jpg, GET logo.png etc.), no google ad server should be hit. Now you could argue that bots could inflate the popularity of a website and therefore the cost to run ads on it. But I guess websites that show ads have most likely google analytics running, one of the only ways Google can actually calculate the popularity (besides Google Search and maybe Chrome history). So it should be no problem for Google to exclude bots from the popularity calculation by analyzing traffic. Maybe I am just missing something, I am also no ad expert at all.

anchovy_ 3y ago
Don't nail me down on this but I think since nowadays' websites are often dynamic, you most likely have to employ headless browsers in order to do whatever it is you want to do. This should then result in fake impressions.
- multicast 3y ago
  Good point. Haven't thought about that.
jsnell 3y ago
It's not about bots creating fake ad impressions by accident. It's people writing bots whose purpose is to fake ad impressions and clicks. They'll then run it on their own website that's running ads, with the goal of being paid by the ad network for this fake traffic.
- multicast 3y ago
  But isn't this a win situation for Google to a certain extent? Since it uses up the budget of the advertiser much faster. And the accuracy of filtering new revenue coming from ads as a company is already fairly limited in general. But maybe there are multiple reasons that Google really only wants to serve real humans to the ads of its clients.

ryu2k2 3y ago

I don't want to play devil's advocate, but when I consider the history of personal computing it would almost surprise me if this didn't get through. From locked-down smartphones having become the norm to windows having turned into a carrier for advertisements, this just seems like the next step in some inevitable evolution of IT. Maybe news agencies aren't much interested because this is still only an early proposal, but considering its implications I find it striking how little it seems to be talked about on the web (outside of hackernews). Rossmann seems to be the only one with a video on the topic up on Youtube. There's only a handful of Twitter results for "web environment integrity api" with next to no replies to them. When I look the keystring up on reddit the only result with a noteworthy amount of debate is not related to WEI at all. Social media is probably just on its last legs, but it doesn't seem like too many people that care are left to fight for what the web, or even general computing, used to be.

keybits 3y ago

Related to WEI, Apple already has something similar live for Safari users, see: "Apple already shipped attestation on the web, and we barely noticed" - https://httptoolkit.com/blog/apple-private-access-tokens-attestation/ https://httptoolkit.com/blog/apple-private-access-tokens-att...

Young-Lord 3y ago

Let's take action now. https://github.com/Young-Lord/fight-for-the-open-web https://github.com/Young-Lord/fight-for-the-open-web