10 ms·

Wow, what an awesome dive into some of the technical aspects behind one of my favorite tools for using the web. And I do think of it that way these days, it's f

by xoa 6y ago

Wow, what an awesome dive into some of the technical aspects behind one of my favorite tools for using the web. And I do think of it that way these days, it's fairly stunning on some sites to switch off all the block and see how they become genuinely unbrowsable. I remember seeing Gorhill discuss a few times over the years some of the reqs for uBO during certain times (like why it could no longer work with Safari following changes Apple made a while back), but so cool to have it all collected in one place. Having said that I've also been fairly stunned recently to see how much difference a simple DNS blacklist system can make too. Not because it's a big technical achievement but because it isn't and in principle seems relatively trivial to work around. But as I've been switching all my routing from UniFi to OPNsense, I've gone ahead and tried out Unbound's basic built-in blacklisting. While it's no uBO, it works on every single device and browser including in apps and it seems like it really shouldn't, that more parties would just be proxying ads through their own infra and DNS. Been kind of an interesting illustration of technical vs economic influences in an ecosystem. I can see how proxying would add complexity and cost to setup so it must just be that few enough people do it the ad industry can't be bothered. But should that ever catch on (and it could, Raspberry Pi seems fairly well known) I expect uBO to be able to keep up with the cat-and-mouse long after DNS has been left behind. This piece helps underline how incredibly important maintaining a critical level of diversity in the browser ecosystem is. Just shortly ago there were a bunch of complaints again about Apple not allowing Chrome to be on iOS because it "holds back the web", but what "holding back the web" looks like is certainly a matter of perspective...

estaseuropano 6y ago
I guess the issue with proxying is that the ad provider has less control/data and can't be sure whether views are genuine.
- jtbayly 6y ago
  That’s great if you’re right, because it means it is a fundamental limitation that can’t easily be worked around.
  - chillfox 6y ago
    They will work around it by being your DNS provider and proxy for your site.
- chillfox 6y ago
  Unless the ad provider proxy the site instead of the ads. That way they get more control/data.
  - collinmanderson 6y ago
    like AMP
  - kevin_thibedeau 6y ago
    It's only a matter of time before someone develops a wasm browser engine that renders to canvas and provides "trusted" delivery of ad assets over a websocket.
    - jhugo 6y ago
      How could that be any more “trusted” than doing it without wasm and websocket?
    - adrianN 6y ago
      As long as the user still has control over their machine (i.e. the browser) this approach doesn't work very well. You can't really run trusted code on an untrusted machine.
      - rini17 6y ago
        No, it does work well. When they deliver the website as obfuscated binary code, it's much harder for user to change its behavior. Notice parent quoted the word "trusted" so you disagree with something other than they meant.
    - account42 6y ago
      That would be nice since you could just block canvas by default and only allow it for a few site that have legitimate use for it.
1vuio0pswjnm7 6y ago
Funny, I see the possible future differently. I am inclined to predict that the option to use ad blockers may come to an end because browsers and other software financed with online ad-derived revenue have full control over what extensions users are allowed to run. They have full control over that system for extending the functionality of their software. Whereas the system for supplying software with IP addresses should continue to remain controllable by the user.^1 The online ad ecosystem has never had full control over that system. 1. If they so choose to exercise said control. As cited, probably most users are not currently exercising that control. This allows online ad tech to operate with relative ease, at relatively lower cost. As much as anyone loves their ad blocker (to be clear, I think they are great), we really cannot dismiss the role of DNS in ad blocking. After all, it is DNS that is being used to bypass uBO on browsers other than Firefox. The only reason uBO can achieve a 70% success rate^2 blocking CNAME-cloaked ad/tracking on Firefox is because Firefox has a "DNS API". As such, uBO can check the results of DNS lookups for ad/tracking server hostnames/IPs.^3 Of course, a user who blocks ads/tracking outside the browser by controlling her own DNS lookups also has access to those results. No API needed. (Although I think it's a great project, I personally do not use Pi-Hole. I was using DIY DNS (without dnsmasq) long before Pi-Hole.) One solution I can see to a future where ad blocking extensions are banned is a user-controlled proxy that performs similar operations, but outside the browser. (I make most HTTP requests for recreational web use outside the browser, through a proxy I control. For recreational web use, I do not use a major browser to make HTTP requests to the proxy. The programs that I use have no financial dependence on online advertising/tracking/data collection like the major browsers and many mobile apps do.) 2. According to the paper cited on Github page that comprises the OP. 3. According to some uBO users sometimes the uBO-triggered lookups are undesired, e.g., when using proxies/VPNs.
- MetaWhirledPeas 6y ago
  > I am inclined to predict that the option to use ad blockers may come to an end because browsers and other software financed with online ad-derived revenue have full control over what extensions users are allowed to run. They have full control over that system for extending the functionality of their software. I have a feeling that if Chrome (for instance) went that route it would quickly become Internet Explorer'd. If you recall, IE was the Chrome of its day for a good while until it slowly eroded away all its goodwill.
AviationAtom 6y ago
You should checkout NextDNS
throwaway189262 6y ago
I think the future is machine learning based blocking. Ads are obvious, they have to be for users to see them. You could probably use text classification and object recognition to filter ads effectively. And you could do it from the view layer where nothing on the page can tell they've been blocked. This is also my tinfoily theory for why Chrome restricts the API's used by ad blockers. It's to prevent more effective blockers from being developed.
- dbt00 6y ago
  I don't think so. Machine learning is generally poorly suited to dealing with an intelligent adaptive adversary capable of an unpredictable universe of inputs.
  - throwaway189262 6y ago
    Text classification might be very good at identifying ad text. There's a popular fake news dataset where models hit 85% accuracy https://link.springer.com/chapter/10.1007/978-3-030-68787-8_45 https://link.springer.com/chapter/10.1007/978-3-030-68787-8_... Ads are highly tuned for click through rates. Even if the model isn't 100% accurate, it would force advertisers to use less effective ad text to avoid filters.
    - lone-commenter 6y ago
      > it would force advertisers to use less effective ad text to avoid filters. That seems counterproductive to me. I mean, ads would still be annoying. And that’s why we block them.
      - mrob 6y ago
        I decline to accept ads because I don't want professional manipulators tricking me into acting against my best interests. I consider myself fairly good at resisting manipulation, but these are highly skilled experts backed by the full power of modern neuroscience, and they aim to catch me off guard. Adverts are hazardous to view, and you should not do so without utmost mental focus and discipline, which is impractical during general browsing. The avoidance of annoyance is a just bonus.

by xoa 6y ago

estaseuropano 6y ago

I guess the issue with proxying is that the ad provider has less control/data and can't be sure whether views are genuine.

jtbayly 6y ago
That’s great if you’re right, because it means it is a fundamental limitation that can’t easily be worked around.
- chillfox 6y ago
  They will work around it by being your DNS provider and proxy for your site.
chillfox 6y ago
Unless the ad provider proxy the site instead of the ads. That way they get more control/data.
- collinmanderson 6y ago
  like AMP
- kevin_thibedeau 6y ago
  It's only a matter of time before someone develops a wasm browser engine that renders to canvas and provides "trusted" delivery of ad assets over a websocket.
  - jhugo 6y ago
    How could that be any more “trusted” than doing it without wasm and websocket?
  - adrianN 6y ago
    As long as the user still has control over their machine (i.e. the browser) this approach doesn't work very well. You can't really run trusted code on an untrusted machine.
    - rini17 6y ago
      No, it does work well. When they deliver the website as obfuscated binary code, it's much harder for user to change its behavior. Notice parent quoted the word "trusted" so you disagree with something other than they meant.
  - account42 6y ago
    That would be nice since you could just block canvas by default and only allow it for a few site that have legitimate use for it.

1vuio0pswjnm7 6y ago

Funny, I see the possible future differently. I am inclined to predict that the option to use ad blockers may come to an end because browsers and other software financed with online ad-derived revenue have full control over what extensions users are allowed to run. They have full control over that system for extending the functionality of their software. Whereas the system for supplying software with IP addresses should continue to remain controllable by the user.^1 The online ad ecosystem has never had full control over that system. 1. If they so choose to exercise said control. As cited, probably most users are not currently exercising that control. This allows online ad tech to operate with relative ease, at relatively lower cost. As much as anyone loves their ad blocker (to be clear, I think they are great), we really cannot dismiss the role of DNS in ad blocking. After all, it is DNS that is being used to bypass uBO on browsers other than Firefox. The only reason uBO can achieve a 70% success rate^2 blocking CNAME-cloaked ad/tracking on Firefox is because Firefox has a "DNS API". As such, uBO can check the results of DNS lookups for ad/tracking server hostnames/IPs.^3 Of course, a user who blocks ads/tracking outside the browser by controlling her own DNS lookups also has access to those results. No API needed. (Although I think it's a great project, I personally do not use Pi-Hole. I was using DIY DNS (without dnsmasq) long before Pi-Hole.) One solution I can see to a future where ad blocking extensions are banned is a user-controlled proxy that performs similar operations, but outside the browser. (I make most HTTP requests for recreational web use outside the browser, through a proxy I control. For recreational web use, I do not use a major browser to make HTTP requests to the proxy. The programs that I use have no financial dependence on online advertising/tracking/data collection like the major browsers and many mobile apps do.) 2. According to the paper cited on Github page that comprises the OP. 3. According to some uBO users sometimes the uBO-triggered lookups are undesired, e.g., when using proxies/VPNs.

MetaWhirledPeas 6y ago
> I am inclined to predict that the option to use ad blockers may come to an end because browsers and other software financed with online ad-derived revenue have full control over what extensions users are allowed to run. They have full control over that system for extending the functionality of their software. I have a feeling that if Chrome (for instance) went that route it would quickly become Internet Explorer'd. If you recall, IE was the Chrome of its day for a good while until it slowly eroded away all its goodwill.

AviationAtom 6y ago

You should checkout NextDNS

throwaway189262 6y ago

I think the future is machine learning based blocking. Ads are obvious, they have to be for users to see them. You could probably use text classification and object recognition to filter ads effectively. And you could do it from the view layer where nothing on the page can tell they've been blocked. This is also my tinfoily theory for why Chrome restricts the API's used by ad blockers. It's to prevent more effective blockers from being developed.

dbt00 6y ago
I don't think so. Machine learning is generally poorly suited to dealing with an intelligent adaptive adversary capable of an unpredictable universe of inputs.
- throwaway189262 6y ago
  Text classification might be very good at identifying ad text. There's a popular fake news dataset where models hit 85% accuracy https://link.springer.com/chapter/10.1007/978-3-030-68787-8_45 https://link.springer.com/chapter/10.1007/978-3-030-68787-8_... Ads are highly tuned for click through rates. Even if the model isn't 100% accurate, it would force advertisers to use less effective ad text to avoid filters.
  - lone-commenter 6y ago
    > it would force advertisers to use less effective ad text to avoid filters. That seems counterproductive to me. I mean, ads would still be annoying. And that’s why we block them.
    - mrob 6y ago
      I decline to accept ads because I don't want professional manipulators tricking me into acting against my best interests. I consider myself fairly good at resisting manipulation, but these are highly skilled experts backed by the full power of modern neuroscience, and they aim to catch me off guard. Adverts are hazardous to view, and you should not do so without utmost mental focus and discipline, which is impractical during general browsing. The avoidance of annoyance is a just bonus.