8 ms·
Ug, I've been railing against this for a bit now. For anyone who is unclear what is happening, Chrome downloads a set of partial hashes for sites that are consi
by kodablah 9y ago
Ug, I've been railing against this for a bit now. For anyone who is unclear what is happening, Chrome downloads a set of partial hashes for sites that are considered bad by this coalition. They don't provide the full list of sites (evil). Then if the site matches it, it phones home to get the full hash to see if it matches that. If it does, it applies the EasyList (non-cosmetic) ad block rules.
Problems:
* The list is totally opaque (I am wrong, see EDIT 2 below)
* They use the hard work of people like EasyList and subjectively apply it (though not that big of a deal, they do make it free/open for all uses after all)
* They build it into the browser instead of as an extension or working with the existing ad-block community
I urge everyone to keep with uBO and the like. How anyone can be for NN and then think a coalition can be an on-by-default gatekeeper of good or bad web items I'll never understand. At this point, I have a hard time separating browser from ISP wrt end user control and limited choice (especially for the masses who aren't familiar w/ these kinds of details).
EDIT: I should note that this is the same mechanism by which the safe browsing lists work that tell you a page may be bad. For consistency, I disagree with that too of course, but I find the motives and targets here to be much more sinister. I would also say switch to FF, but they also use the secret safe browsing lists, so they'll probably switch to this as well. I say find a Chromium/Gecko based browser w/ all the ancillary shit like this removed.
EDIT 2: There is a method of obtaining the entire list via the API, see comments below. I was wrong about the opacity and stand corrected. Still doesn't alleviate the concerns around gatekeeping. I wonder if Google would let me keep a running update of this list in GitHub so we can all watch changes and other things like adblockers could use it.
- quadrature 9y agoDo you have a source for this?, not disputing you, just want to read up on this.
- kodablah 9y agoNot really, just based on the blog post and comments at https://news.ycombinator.com/item?id=16297550 https://news.ycombinator.com/item?id=16297550 and elsewhere where myself and others looked at the code. I could be wrong on some aspects of course and would welcome being corrected. Also, I cannot find a clear explanation from Google how the internal details of this ad blocker work (how they manage the hashes on their side, where this coalition list is maintained, etc). I would love a link, but alas with these kinds of things transparency is usually the first thing to go, especially since people don't demand it.
- colonelxc 9y agoThe Safe Browsing distribution is for efficiency (less bandwidth, less in-memory data to store). The Better Ads Standard list can be obtained here (unhashed): https://developers.google.com/ad-experience-report/ https://developers.google.com/ad-experience-report/
- kodablah 9y agoEfficiency is distributing the entire list (and they do distribute an indexed and unindexed full EasyList, you can see it in your user's Chome data folder). I doubt it's too big to download in indexed form for a desktop and keep updating with deltas. At the least, I'd like the option. EDIT: Updating from previous statement saying I couldn't find where to get the list. I have now obtained it from [0] and put it at [1] (caution, it's a large gist). 0 - https://developers.google.com/ad-experience-report/v1/reference/rest/v1/violatingSites/list https://developers.google.com/ad-experience-report/v1/refere... 1 - https://gist.github.com/cretz/18594176f791fc0ede26078f76cf1202 https://gist.github.com/cretz/18594176f791fc0ede26078f76cf12...
- evmar 9y ago(Disclaimer: haven't worked on Chrome in ~7 years.) This weird hashing scheme comes from safe browsing (which blacklists sites that install malware etc.). I guess (without specific knowledge of it) it was just reused for this ads thing because they had all the code handy for it, both the browser-side code and serving code. For safe browsing, as I recall the data format was designed with Mozilla -- that tech predated the existence of Chrome. There's some history about it here: https://wiki.mozilla.org/Security/Safe_Browsing https://wiki.mozilla.org/Security/Safe_Browsing and https://wiki.mozilla.org/Phishing_Protection:_Design_Documentation https://wiki.mozilla.org/Phishing_Protection:_Design_Documen... I recall the weird hashing scheme was carefully designed to balance some concerns. For example when it phones home, it phones home with a hash of the current URL so that it doesn't reveal the current URL to the server (unless the URL is already in the server-side blacklist). I also think it was intentional that the client didn't get a list of all known-malware URLs. I can't find any design docs for it at the moment better than https://developers.google.com/safe-browsing/v4/ https://developers.google.com/safe-browsing/v4/ . It may well be the case that the hashing scheme doesn't make sense at all in this context.
- cfitz 9y agoWhile I would also personally prefer this functionality as an extension rather than as "bloatware" (although it's not exactly useless), I believe having it built-in will better the majority of Chrome users who are not savvy (or aware) enough to install extensions, which as of Dec 2010 was only 1/3 of Chrome users (the most recent stats from Chromium I can find) [1]. [1] https://blog.chromium.org/2010/12/year-of-extensions.html https://blog.chromium.org/2010/12/year-of-extensions.html
- kodablah 9y agoUnlike some others, I don't doubt the good intentions this road is paved with. But to keep with the NN analogy, less savvy users would also prefer paying for a cheaper, more restricted internet.
- SquareWheel 9y ago>secret safe browsing lists Isn't there an open API for querying this information, as well as their webapp? It can't be all that secret. https://transparencyreport.google.com/safe-browsing/search https://transparencyreport.google.com/safe-browsing/search
- kodablah 9y agoOnly one site at a time. I want to see them all. Other ad blockers let me do this with their lists. Google is no stranger to supporting big list downloads. They are choosing not to and should not be defended. Sure they can say they don't want the offenders to know they are offending, but they can find that out anyways. I think in reality, they want to maintain the keys to the lists which means not giving it away and keeping it secret.
- SquareWheel 9y agoThat's assuming that it's a simple process to publish a huge list. Keep in mind that Safe Browsing is much more complex than an adblock list. They allow actions from site owners via Search Console, and track many types of threats (phishing sites, malicious downloads). My guess would be there isn't a simple canonical list, and it's more of a heuristic evaluation where certain thresholds will trigger site warnings. In this case an API would be more up-to-date, and less computationally expensive than serving large, always-changing lists. It was likely just the more logical choice. Though I'm assuming the new ads functionality uses the same Safe Browsing infrastructure. >Sure they can say they don't want the offenders to know they are offending Actually not the case at all. They send alerts to sites that are affected by Safe Browsing via Search Console.
- reaperducer 9y agoWe all know that having Google as gatekeeper has never pushed innocent, content-heavy, legitimate web sites into oblivion before. Oh, wait...
- lev99 9y agoAm I missing an obvious example?
- forgotpw2019 9y agoI'm not sure if you're a Google PR drone or just naive, but people have been talking about this for years... Ya know the nice little extensions Google has added over the years to their search page? Like speed test, recipes, dictionary, a rudimentary calculator, map locations, etc?? Most of the information is stuff they stole from other websites. They put it in the search results now so you won't go to those terrible websites yourself and stop looking at Google ads. And nobody can stop it because if you block Google spiders your website dies. So the only people that can fight Google stealing all their shit are other Titans like Facebook and LinkedIn. Everyone else is getting their content lifted and moved into search results. It's so pervasive that I'm not even going to bother to find you links. Ironically, you can just Google something like "Google destroyed my business" and find plenty of results. Google does a lot of things I support but they also destroy businesses intentionally with reckless abandon. For instance, I find it impossible to believe Google didn't think it would destroy Speedtest.com when it added a widget to the search page. Or Allrecipes when they started stealing their content and ploping it on the search page. Some of the things Google does these days are thinly veiled as being for "convenience" or "security" when they're really just a monopoly stomping out various competitors. Remember how Google used to punish "cloaking"? Well now they quietly allow you to do whatever the hell you want which led to all the news site paywalls. These sites we're tired of Google stealing all their shit and powerful enough as a group to stand up to it. Google sits on it's hands because they know if they stop it they will end up in court, probably for antitrust
- jankeymeulen 9y ago
- redm 9y agoMy problem is that Google was a founder of Better Ads and its designed to 1) protect their ad business, 2) stop people from wanting ad blockers, and 3) make the web nicer. Huge emphasis on #1. Whenever you have the browser maker, and punitive actions controlled by the same party, and arbitrarily, its a recipe for disaster. If Google really cared, they should spin Chrome off to a foundation, provide it a large amount of funding, and totally step aside. Having the #1 web browser and the largest ad network, controlled by Google, even if you agree with what they are doing, is a recipe for disaster. Google, of course, MUST protect its ad business, let's call his what it is.
- eli 9y agoSo because of who they are Google should not be allowed to attempt to improve the quality of ads on the internet? Why would Chrome being run a foundation that is funded overwhelmingly by Alphabet make a difference? Chromium is already public -- aren't there already forks that focus on privacy and ad blocking?
- maxxxxx 9y agoYou don't see a massive conflict of interest here?
- eli 9y agoSure, I just don't see why that should be disqualifying. If they are too shy about blocking ads, then most users will not even perceive that this feature exists and the status quo is unchanged. If they block even a handful of the worst offenders, it's a win for Chrome users. And if those blocks encourage people to clean up their act then it's a win for everyone.
- maxxxxx 9y agoI don't know. Maybe there is still a small benefit but I think their main reason for participation is to steer the effort into a favorable to them direction. Some benefits will come out of it but any real progress will be blocked or massively watered down because it would cut into their bottom line.
- coliveira 9y agoThis seems like asking the fox to guard the chickens.
- dclowd9901 9y agoRight now, per your metaphor, all the chickens are being eaten. It's a clusterfuck out there with some of these ads. But I also don't want to go the opposite way and (to extend the metaphor) have a fox so vicious, it keeps _myself_ away from the chickens (a la AdBlock extensions), such that my favorite websites go away because I'm not supporting their revenue stream. Google does not have an interest in showing you shitty ads. They have an interest in providing their clients with ads that are acted upon and that web users appreciate, and they have an interest in having a browser that does that with as little friction as possible. I'd say their incentives are perfectly aligned with ours.
- stordoff 9y agoRight now, all of the chickens are being eaten. If we get a fox that only occasionally eats a chicken, but keeps the other more vicious foxes away, that's a net win.
- freehunter 9y agoThey have an interest in showing you non-shitty ads. They sell non-shitty ads. Therefore they have an interest in showing you their ads. They also own the most popular way those ads are shown: through their web browser. This browser has the ability (at Google's sole discretion) to block shitty ads, of which all of them are run by their direct competitors. So what's happening is, Google is giving themselves the right to block parts of the Internet they don't like, with the implied statement of "if you want your ads to be seen by Chrome users, buy them from us or else we might block them". That is a classic protection racket. "Want your ads to be seen? Better buy them from us, or... bad things will happen."
- euyyn 9y agoAnother user posted that the criteria for shittiness is defined here: https://www.betterads.org/standards/ https://www.betterads.org/standards/ Seems pretty reasonable.
- cptskippy 9y ago> I should note that this is the same mechanism by which the safe browsing lists work that tell you a page may be bad. For consistency, I disagree with that too of course, but I find the motives and targets here to be much more sinister. Out of curiosity what is your objection to the Safe Browsing lists? I personally have my router blocking sites at the IP level and DNS rejecting queries for domains off this list. I can't remember the last time I couldn't visit a site because it was on the list but with the prevalence of zero-day malware and naive users on my network, I'd rather just not even deal with the potential.
- kodablah 9y ago> Out of curiosity what is your objection to the Safe Browsing lists? The gatekeeping and phoning home (even though it is privacy conscious). But it's not a strong objection. In general I prefer browsers to be neutral by default and make no HTTP requests behind the scenes, but I acknowledge that's unreasonable for most users. It's less about my personal objection and more about an objection to on-by-default corporate decision making being deployed to millions of users. While safe browsing hasn't come under much scrutiny due to its limited scope and that it hasn't been abused, I suspect it won't be long before someone's site has its ads blocked unfairly by the coalition. I understand with human review and pending-vs-actual-blockage incubation they are attempting to alleviate false positives, but the internet is too large IMO and the rules are subjective (so I can have a site with a 29% ad density?).
- xori 9y agoOn desktop, I agree with you. But I hope they do this for mobile, browsing on android is a dangerous place on ad-ridden websites.
- JoshMnem 9y agoDownload Firefox for Android. It's the only mobile browser I know of that lets you install extensions.
- r3bl 9y ago> I would also say switch to FF, but they also use the secret safe browsing lists, so they'll probably switch to this as well. They use this by default, but you're free to turn it off, and that setting (unlike others) seems to sync just fine using Firefox Sync. Just checked and it's off. I turned it off once years ago and switched between OSs and different devices in the meantime, and it's still off. Don't know if the same is possible on Chrome. > I should note that this is the same mechanism by which the safe browsing lists work that tell you a page may be bad. For consistency, I disagree with that too of course, but I find the motives and targets here to be much more sinister. I agree with you completely on this one. I had to turn that off on two layers, on a browser level, and inside my antivirus' firewall. My antivirus now constantly complains that I'm "not fully protected" because I don't want it to do web filtering for me. I do support other private browsing techniques that stay on my device, like first party isolation, and adblock-like lists that are downloaded on my device. Both are available in Firefox, and using them doesn't collide with my stance on NN, since I'm the one preventing the content from being fetched by my machines.
- Ajedi32 9y ago> The list is totally opaque (I am wrong, see EDIT 2 below) As you noted, this is false. See https://developers.google.com/ad-experience-report/ https://developers.google.com/ad-experience-report/ > They use the hard work of people like EasyList and subjectively apply it (though not that big of a deal, they do make it free/open for all uses after all) Again, I don't see the problem here. As you said, EasyList is free and open. It seems like your objection is just that you don't like the way its being used? > They build it into the browser instead of as an extension or working with the existing ad-block community I'd actually consider this a huge win. Everyone gets it by default (though you can turn it off in settings if you want), and it works on mobile, which doesn't currently allow extensions. What's your actual concern here? > think a coalition can be an on-by-default gatekeeper of good or bad web items I'll never understand. At this point, I have a hard time separating browser from ISP wrt end user control and limited choice There's a reason browsers are called the "user agent"; it's because their purpose is to make decisions and perform actions on behalf of the user. If a browser vendor wants to block ads on behalf of the user on sites which they deem to be using those ads in an abusive manner (and the standards for what is considered "abusive" in this case are actually pretty clear-cut: https://www.betterads.org/standards/ https://www.betterads.org/standards/) I don't see a problem with that. If you as a user don't want your user agent behaving that way, you can either tell it to stop (chrome://settings/content) or get yourself a new user agent. (And both of those actions are significantly easier than they are with your ISP I might add.)
- kodablah 9y ago> Again, I don't see the problem here. As you said, EasyList is free and open. It seems like your objection is just that you don't like the way its being used? Not really my objection, I just wonder if it's the objection of the people putting in the work on those lists. I personally don't think it's a big issue. > I'd actually consider this a huge win. Everyone gets it by default (though you can turn it off in settings if you want), and it works on mobile, which doesn't currently allow extensions. What's your actual concern here? A bigger win would be to allow mobile extensions. It's strange to use a decision Google makes as a reason Google has to do something this way instead, ha. That it's on by default makes those of us on the non-user side of the web wary of the slippery slope of browsers not being neutral about what is shown to our users. > I don't see a problem with that. If you as a user don't want your user agent behaving that way, you can either tell it to stop (chrome://settings/content) or get yourself a new user agent. I'm more concerned with the site developer side than the user side. If something ships to millions of users and begins to exercise non-neutral control over content, you should become concerned. It's like TVs constantly updating a list of shows they won't allow to be shown on their TV. You might tell a user to go change the settings of the TV, but as a someone making the video, would you not be concerned? Or glasses that limit some of what you get to see by default, or headphones that disable some of what you get to hear by default, or cars that don't allow you to enter some areas by default, etc. I hope it's clear that conduits like these need to remain neutral.
- drawnwren 9y agoFor anyone curious why Google chose to implement it this way, it's because they're using a bloom filter for the first check. This is much more space efficient than storing the entire list in memory.