9 ms·

At some point I really think we'll resort to blocking ads at the network level. There's no way to detect a block at that level and we'll truly be able to contro

At some point I really think we'll resort to blocking ads at the network level. There's no way to detect a block at that level and we'll truly be able to control the content that way. I've seen a few http/dns ad blockers here on HN already so we know it's possible and being thought about. These browser based detections are just a game of cat and mouse at this point.

johnmaguire2013 11y ago
This is already how it's generally done on Android (via hosts file.) Still detectable though: You can detect broken images on the page for example, which would tell you that the hosts have been blocked.
- Spivak 11y ago
  I would imagine that the cost of a false positive if something goes wrong with the ad network would be pretty damaging. I don't know if it's still the case but Comedy Central's video player was atrocious, the video wouldn't play unless it successfully played an ad but ads just straight up failed to load 30/40% of the time so you had to refresh, watch an ad to get to the video controls, scan back to your place, watch another ad because you crossed a 'commercial break' point, have that fail 10 minutes later, and repeat. The whole experience was awful and lead to me just scraping the site for the direct steam link and eventually just deciding that Jon Stewart wasn't worth it.
  - yareally 11y ago
    I've block ads with a host file on PC for 10 years. Maybe it's my browsing habits, but I can count the number of times it affected my ability to browse content on one hand. I also manually add a lot of the entries so imagine that also helps.
  - forgingahead 11y ago
    Jon Stewart wasn't worth it This is the key takeaway here
  - dredmorbius 11y ago
    Pretty much exactly my experience with CC, and that despite being a strong admirer of Stewart. HBO is following a vastly saner path with John Oliver, whose videos simply appear on YouTube. I'm not generally a fan of single-vendor consolidation, but if: 1. Video providers aren't going to directly provide media and... 2. YouTube Just Works (including via scrapers such as youtube-dl), well, then so it goes. NB: ytdl also hits a ton of other sites, including Vimeo and The Internet Archive (which, truth told supports direct download and streaming, but ytdl is in finger memory now). Those that Just Work will be utilised.
  - smcl 11y ago
    Ah I'm glad to find out it wasn't just me that suffered through this. Actually I genuinely wouldn't have minded so much, if it wasn't exactly the same ads day after day. I can recite that new Comedy Central App advert word-for-word (complete with everything each comedian says - "This is the height of luxury!", "Uh-oh ha-ha", "Give it to me", "VISIT ME!" etc)
- drdeca 11y ago
  I guess you could have something that makes it still successfully retrieve a working image, just not the actual ad maybe? I guess then an ad-versary could check if the images are signed or w/e?
logicallee 11y ago
you're not entitled to have some server serve you whatever content you want it to, it doesn't have to answer your http requests - so logically, by whatever means we get there, the end-game is for servers to monitor and exclude you. but the status quo is a lot better now, than for what it would take for that to happen. Do you want servers to figure out that you're blocking ads at the network level, and tell each other not to serve to your IP? Because it is a lot less wrong for them to do it - they have a lot more right to do it, if it's in their TOS - than for you to recraft their content so that you're only making http requests to contents, but not their ads. It's like going grocery shopping for milk and sugar at restaurants and coffee places, i.e. because they have it out 'for free'. you might say, hey, they're making it available for free, it's not your fault if they have a broken business model. But it's their right whether they want you there. The analogy isn't perfect, it's quite leaky, so let's get back to the technical facts here. The cat and mouse game is between a server wanting to serve content as well as ads, and some consumers wanting to recraft the requests so their clients/browsers do not load ads. Since ultimately the server has what the users want, for example articles, and the business model is some limited part of the attention of the user, I don't see this ending well for users who want to consumer the content but do not share any part of their attention with ads. It's just not going to work out.
- rpgmaker 11y ago
  Interesting point. I will say that, if in the end the user will have to accept the ads anyway because all the content providers will refuse them otherwise then I better enjoy ad blocking while I can :-)
  - logicallee 11y ago
    people don't get my point. The future I describe is technically possible, and within servers' rights, but let's not have it come to that. Let's stop here: with the status quo. Playing cat-and-mouse with a server's ad-block detector and with its choice not to serve users who block its ads is going too far. It would be as though the server requested the user the actively agree on a screen, to seeing tasteful on-topic and non-intrusive ads, and the user must agree to continue: only to have an ad-blocker remove that screen but send the message "I agree to see advertising and promise to add this site to the whitelist" without doing so, as a lie and a forgery. While this would technically circumvent this measure, it is ridiculous for an ad-blocker to forge that agreement. (I am making an off-topic analogy, this is not related to what is happening now.) Likewise, this is going too far. It is morally wrong and technically stupid to go down this rabbit-hole, because technically, legally, and morally it leads to exactly what I have described. Let's stop here.
orionblastar 11y ago
I used to have a program that blocked IP addresses of advertising sites and replaced all pictures with any picture I wanted to use. I used a skull and crossbones instead so in place of any ad I had a pirate flag instead. I forgot the name of the program that did that, it was like some sort of firewall that redirected to 127.0.0.1 and ran a web server that served up any image I wanted for the picture. I used to add IP addresses to my hosts file to block advertising sites as well. Now I just use uBlock but they can detect that. I don't mind advertising as much as long as it doesn't get annoying with pop-ups and other crap. It should not, for example, open up another browser window for me to display an ad.
- mistaken 11y ago
  Perhaps he was using ettercap: http://www.irongeek.com/i.php?page=security/ettercapfilter http://www.irongeek.com/i.php?page=security/ettercapfilter
- deleted 11y ago
  [deleted]
- escap 11y ago
  Was it http://www.privoxy.org/ http://www.privoxy.org/ ? e.g. http://www.privoxy.org/user-manual/quickstart.html http://www.privoxy.org/user-manual/quickstart.html
  - orionblastar 11y ago
    It sounds like it was, but I am not 100% sure.
chris_wot 11y ago
That's not going to work if everything runs via https. How exactly do most ad-blockers work? Are they searching the DOM and blocking elements from showing? Or can they see that there are asychronous requests being made from JavaScript to known ad networks? I'm musing out loud here, but take for instance the Sydney Morning Herald - the sheer number of trackers is extraordinary. The full load of just the main page, and images are reasonably light, is about 8MB... and there are literally dozens of trackers. How do ad-blockers prevent these from appearing?
- uxp 11y ago
  They usually bind to events the browser triggers before a network request is made. This lets them literally block the browser from downloading any content from a source that may register your IP just from the connection (pixel tracking, etc). Searching the DOM and removing elements would still require a network request to obtain the content before realizing that it's a tracked element. By blocking the request from happening, we can truly say that an Content Blocker on iOS (for instance) speeds up the browsing experience because the request isn't even initiated.
zuron7 11y ago
It is possible. Not sure how, but my school used to block all ads because of the limited bandwidth available. 50Mbps divided among ~1500 people with atleast 100 using it any given point.
silverwind 11y ago
A pretty good way to do this is to put something like http://someonewhocares.org/hosts/zero/ http://someonewhocares.org/hosts/zero/ in your router's host file and enable DNS intercepting. It won't catch sneaky same-domain ads but it's great in that it consumes no extra resources on the end device and also works on iOS devices.
- spookyuser 11y ago
  would this be more effective than something like adblock.sh https://github.com/teffalump/adblock https://github.com/teffalump/adblock
- pgl 11y ago
  Or: http://pgl.yoyo.org/adservers/serverlist.php?hostformat=bindconfig http://pgl.yoyo.org/adservers/serverlist.php?hostformat=bind...
joenathan 11y ago
I'm running Tomato firmware on my router(ASUS RT-AC68P) using an adblocking script. It will resolve known ad servers IP's to itself and serve a 1x1 transparent png instead of the ad. It's a combination of a DNS blacklist and pixelserv. All the devices on my network are ad free.
- atmosx 11y ago
  I run a modified version of PiHole which does the same. Doesn't block 100% of ads, but it blocks enough to keep me happy. I implemented DNSSEC + DNSCrypt just for sake of it too.
- deleted 11y ago
  [deleted]
- Buge 11y ago
  What about https ads?
  - yareally 11y ago
    If I were going with blocking at the router, I'd recommending finding a router that has an accessible host file and add them there. Either that or ipset + iptables (not as recommended for ad blocking since you're likely blocking by hostname and not IP). For example (should work on any router with iptables/ipset): https://github.com/RMerl/asuswrt-merlin/wiki/Using-ipset https://github.com/RMerl/asuswrt-merlin/wiki/Using-ipset https://github.com/RMerl/asuswrt-merlin/wiki/Disable-Windows-10-tracking https://github.com/RMerl/asuswrt-merlin/wiki/Disable-Windows... http://www.snbforums.com/threads/adblocking-with-combined-hosts-file.15309/ http://www.snbforums.com/threads/adblocking-with-combined-ho...
  - deleted 11y ago
    [deleted]
  - deleted 11y ago
    [deleted]
  - dogma1138 11y ago
cm2187 11y ago
As we are moving away from open computing platforms into closed, non customizable computers (iOS, Android, even Windows is heading this way), I wonder if it won't be the only solution. Apple, Google or Microsoft want to protect us from third party tracking but how else can we protect ourselves from Apple, Google and Microsoft tracking us? But it is just another game of cat and mouse. Malware already register new domains on the fly. Advertisers will simply do the same. Keeping these host files updated won't be easy.
startling 11y ago
There are plenty of ways to detect ad-blocking at the network level. e.g. if your ad is an image you can just use `onerror`.
- Drakim 11y ago
  Such detection will also trigger when your ad-network is down, so if you say something rude to the user or block the website's content you are gonna have some angry feedback from a lot of people.
  - wutbrodo 11y ago
    I don't think this is too much of a concern. Assuming the ad network doesn't go down very often, you could do something like "Unable to deliver content. This may be caused by an ad blocker". These kinds of "something is wrong either on your end or ours" messages are fairly common across the web and in this case doesn't leave the user feeling wrongly blamed in case of false positive (since the problem "may" be the ad-blocker).
  - nsgi 11y ago
    Saying something rude would be a bad idea anyway.
deleted 11y ago
[deleted]
hannob 11y ago
Actually doing this at the network level is already done by some ad blockers. And it likely is going to cause more problems than it solves in light of https usage. The only thing you can do there is Man-in-the-Middle connections - with all the bad implications this has: https://blog.hboeck.de/archives/874-More-TLS-Man-in-the-Middle-failures-Adguard,-Privdog-again-and-ProtocolFilters.dll.html https://blog.hboeck.de/archives/874-More-TLS-Man-in-the-Midd...
iSnow 11y ago
>There's no way to detect a block at that level Huh, how hard would it be to check via JS if a layer ad really was shown?
freddref 11y ago
You can redirect dns names to localhost in your hosts file. Super simple.

johnmaguire2013 11y ago

This is already how it's generally done on Android (via hosts file.) Still detectable though: You can detect broken images on the page for example, which would tell you that the hosts have been blocked.

Spivak 11y ago
I would imagine that the cost of a false positive if something goes wrong with the ad network would be pretty damaging. I don't know if it's still the case but Comedy Central's video player was atrocious, the video wouldn't play unless it successfully played an ad but ads just straight up failed to load 30/40% of the time so you had to refresh, watch an ad to get to the video controls, scan back to your place, watch another ad because you crossed a 'commercial break' point, have that fail 10 minutes later, and repeat. The whole experience was awful and lead to me just scraping the site for the direct steam link and eventually just deciding that Jon Stewart wasn't worth it.
- yareally 11y ago
  I've block ads with a host file on PC for 10 years. Maybe it's my browsing habits, but I can count the number of times it affected my ability to browse content on one hand. I also manually add a lot of the entries so imagine that also helps.
- forgingahead 11y ago
  Jon Stewart wasn't worth it This is the key takeaway here
- dredmorbius 11y ago
  Pretty much exactly my experience with CC, and that despite being a strong admirer of Stewart. HBO is following a vastly saner path with John Oliver, whose videos simply appear on YouTube. I'm not generally a fan of single-vendor consolidation, but if: 1. Video providers aren't going to directly provide media and... 2. YouTube Just Works (including via scrapers such as youtube-dl), well, then so it goes. NB: ytdl also hits a ton of other sites, including Vimeo and The Internet Archive (which, truth told supports direct download and streaming, but ytdl is in finger memory now). Those that Just Work will be utilised.
- smcl 11y ago
  Ah I'm glad to find out it wasn't just me that suffered through this. Actually I genuinely wouldn't have minded so much, if it wasn't exactly the same ads day after day. I can recite that new Comedy Central App advert word-for-word (complete with everything each comedian says - "This is the height of luxury!", "Uh-oh ha-ha", "Give it to me", "VISIT ME!" etc)
drdeca 11y ago
I guess you could have something that makes it still successfully retrieve a working image, just not the actual ad maybe? I guess then an ad-versary could check if the images are signed or w/e?

logicallee 11y ago

you're not entitled to have some server serve you whatever content you want it to, it doesn't have to answer your http requests - so logically, by whatever means we get there, the end-game is for servers to monitor and exclude you. but the status quo is a lot better now, than for what it would take for that to happen. Do you want servers to figure out that you're blocking ads at the network level, and tell each other not to serve to your IP? Because it is a lot less wrong for them to do it - they have a lot more right to do it, if it's in their TOS - than for you to recraft their content so that you're only making http requests to contents, but not their ads. It's like going grocery shopping for milk and sugar at restaurants and coffee places, i.e. because they have it out 'for free'. you might say, hey, they're making it available for free, it's not your fault if they have a broken business model. But it's their right whether they want you there. The analogy isn't perfect, it's quite leaky, so let's get back to the technical facts here. The cat and mouse game is between a server wanting to serve content as well as ads, and some consumers wanting to recraft the requests so their clients/browsers do not load ads. Since ultimately the server has what the users want, for example articles, and the business model is some limited part of the attention of the user, I don't see this ending well for users who want to consumer the content but do not share any part of their attention with ads. It's just not going to work out.

rpgmaker 11y ago
Interesting point. I will say that, if in the end the user will have to accept the ads anyway because all the content providers will refuse them otherwise then I better enjoy ad blocking while I can :-)
- logicallee 11y ago
  people don't get my point. The future I describe is technically possible, and within servers' rights, but let's not have it come to that. Let's stop here: with the status quo. Playing cat-and-mouse with a server's ad-block detector and with its choice not to serve users who block its ads is going too far. It would be as though the server requested the user the actively agree on a screen, to seeing tasteful on-topic and non-intrusive ads, and the user must agree to continue: only to have an ad-blocker remove that screen but send the message "I agree to see advertising and promise to add this site to the whitelist" without doing so, as a lie and a forgery. While this would technically circumvent this measure, it is ridiculous for an ad-blocker to forge that agreement. (I am making an off-topic analogy, this is not related to what is happening now.) Likewise, this is going too far. It is morally wrong and technically stupid to go down this rabbit-hole, because technically, legally, and morally it leads to exactly what I have described. Let's stop here.

orionblastar 11y ago

I used to have a program that blocked IP addresses of advertising sites and replaced all pictures with any picture I wanted to use. I used a skull and crossbones instead so in place of any ad I had a pirate flag instead. I forgot the name of the program that did that, it was like some sort of firewall that redirected to 127.0.0.1 and ran a web server that served up any image I wanted for the picture. I used to add IP addresses to my hosts file to block advertising sites as well. Now I just use uBlock but they can detect that. I don't mind advertising as much as long as it doesn't get annoying with pop-ups and other crap. It should not, for example, open up another browser window for me to display an ad.

mistaken 11y ago
Perhaps he was using ettercap: http://www.irongeek.com/i.php?page=security/ettercapfilter http://www.irongeek.com/i.php?page=security/ettercapfilter
deleted 11y ago
[deleted]
escap 11y ago
Was it http://www.privoxy.org/ http://www.privoxy.org/ ? e.g. http://www.privoxy.org/user-manual/quickstart.html http://www.privoxy.org/user-manual/quickstart.html
- orionblastar 11y ago
  It sounds like it was, but I am not 100% sure.

chris_wot 11y ago

That's not going to work if everything runs via https. How exactly do most ad-blockers work? Are they searching the DOM and blocking elements from showing? Or can they see that there are asychronous requests being made from JavaScript to known ad networks? I'm musing out loud here, but take for instance the Sydney Morning Herald - the sheer number of trackers is extraordinary. The full load of just the main page, and images are reasonably light, is about 8MB... and there are literally dozens of trackers. How do ad-blockers prevent these from appearing?

uxp 11y ago
They usually bind to events the browser triggers before a network request is made. This lets them literally block the browser from downloading any content from a source that may register your IP just from the connection (pixel tracking, etc). Searching the DOM and removing elements would still require a network request to obtain the content before realizing that it's a tracked element. By blocking the request from happening, we can truly say that an Content Blocker on iOS (for instance) speeds up the browsing experience because the request isn't even initiated.

zuron7 11y ago

It is possible. Not sure how, but my school used to block all ads because of the limited bandwidth available. 50Mbps divided among ~1500 people with atleast 100 using it any given point.

silverwind 11y ago

A pretty good way to do this is to put something like http://someonewhocares.org/hosts/zero/ http://someonewhocares.org/hosts/zero/ in your router's host file and enable DNS intercepting. It won't catch sneaky same-domain ads but it's great in that it consumes no extra resources on the end device and also works on iOS devices.

spookyuser 11y ago
would this be more effective than something like adblock.sh https://github.com/teffalump/adblock https://github.com/teffalump/adblock
pgl 11y ago
Or: http://pgl.yoyo.org/adservers/serverlist.php?hostformat=bindconfig http://pgl.yoyo.org/adservers/serverlist.php?hostformat=bind...

joenathan 11y ago

I'm running Tomato firmware on my router(ASUS RT-AC68P) using an adblocking script. It will resolve known ad servers IP's to itself and serve a 1x1 transparent png instead of the ad. It's a combination of a DNS blacklist and pixelserv. All the devices on my network are ad free.

atmosx 11y ago
I run a modified version of PiHole which does the same. Doesn't block 100% of ads, but it blocks enough to keep me happy. I implemented DNSSEC + DNSCrypt just for sake of it too.
deleted 11y ago
[deleted]
Buge 11y ago
What about https ads?
- yareally 11y ago
  If I were going with blocking at the router, I'd recommending finding a router that has an accessible host file and add them there. Either that or ipset + iptables (not as recommended for ad blocking since you're likely blocking by hostname and not IP). For example (should work on any router with iptables/ipset): https://github.com/RMerl/asuswrt-merlin/wiki/Using-ipset https://github.com/RMerl/asuswrt-merlin/wiki/Using-ipset https://github.com/RMerl/asuswrt-merlin/wiki/Disable-Windows-10-tracking https://github.com/RMerl/asuswrt-merlin/wiki/Disable-Windows... http://www.snbforums.com/threads/adblocking-with-combined-hosts-file.15309/ http://www.snbforums.com/threads/adblocking-with-combined-ho...
- deleted 11y ago
  [deleted]
- deleted 11y ago
  [deleted]
- dogma1138 11y ago

cm2187 11y ago

As we are moving away from open computing platforms into closed, non customizable computers (iOS, Android, even Windows is heading this way), I wonder if it won't be the only solution. Apple, Google or Microsoft want to protect us from third party tracking but how else can we protect ourselves from Apple, Google and Microsoft tracking us? But it is just another game of cat and mouse. Malware already register new domains on the fly. Advertisers will simply do the same. Keeping these host files updated won't be easy.

startling 11y ago

There are plenty of ways to detect ad-blocking at the network level. e.g. if your ad is an image you can just use `onerror`.

Drakim 11y ago
Such detection will also trigger when your ad-network is down, so if you say something rude to the user or block the website's content you are gonna have some angry feedback from a lot of people.
- wutbrodo 11y ago
  I don't think this is too much of a concern. Assuming the ad network doesn't go down very often, you could do something like "Unable to deliver content. This may be caused by an ad blocker". These kinds of "something is wrong either on your end or ours" messages are fairly common across the web and in this case doesn't leave the user feeling wrongly blamed in case of false positive (since the problem "may" be the ad-blocker).
- nsgi 11y ago
  Saying something rude would be a bad idea anyway.

deleted 11y ago

[deleted]

hannob 11y ago

Actually doing this at the network level is already done by some ad blockers. And it likely is going to cause more problems than it solves in light of https usage. The only thing you can do there is Man-in-the-Middle connections - with all the bad implications this has: https://blog.hboeck.de/archives/874-More-TLS-Man-in-the-Middle-failures-Adguard,-Privdog-again-and-ProtocolFilters.dll.html https://blog.hboeck.de/archives/874-More-TLS-Man-in-the-Midd...

iSnow 11y ago

>There's no way to detect a block at that level Huh, how hard would it be to check via JS if a layer ad really was shown?

freddref 11y ago

You can redirect dns names to localhost in your hosts file. Super simple.