9 ms·

I run a network ad-block dns (pihole) and consistently 25-33% of all my network traffic is blocked as ads. It's much more than I ever imagined. Now I'm used to

by warent 6y ago

I run a network ad-block dns (pihole) and consistently 25-33% of all my network traffic is blocked as ads. It's much more than I ever imagined. Now I'm used to a different internet, when I'm using internet off the network it's like WTF is this?

ogre_codes 6y ago
I just have a decent set of ad blockers and the experience is similar. Unfortunately, it often results in weird experiences or I get sites which don't work at all if you have ad blocking.
- Average_ 6y ago
  Well that's interesting. For me 99.5% of websites work perfectly using ublock origin. The only .5% remaining are websites that actively refuse to serve any kind of adblock users, not because it breaks functionality on their site. I don't think I can recall having visited a single website that would have features break unintentionally because of ublock in the past few years.
  - vianneychevalie 6y ago
    Can't agree with you, Dynamics 365 is one of them (it's shit but I've implemented it at work). EDF (French main electricity provider) also breaks for me. That's one example from a big company, and one example with a big user base.
  - ev1 6y ago
    It depends on how many privacy lists you have added, probably. Normal display ads all being blocked is generally fine 99% of the time, but if you care about not being permanently tracked across the internet then there are a couple more domains you have to add - except some sites make it mandatory that those invasive fingerprinting scripts and port scanners run and report back a session, otherwise you're refused login or banned.
  - ogre_codes 6y ago
    I don't think my experience is vastly different from yours. I do get some sites where pop-overs or cookie notifications are blocked but it's not clear and you just can't scroll. I could turn off those blocker settings, but the notifications are annoying enough it's worth it.
  - letitbeirie 6y ago
    The Denver Post just lost my business over this. They have one of those things that scrambles all the words for any user with the audacity to not want to see video+audio ads while reading their newspaper. Is it their content to do what they want with? Sure. Does the same logic apply to the $9 I used to give them each month? You're damn right.
    - ogre_codes 6y ago
      Hard agree. If I'm paying for content, I'd accept a small number of discrete advertising. Video advertising on a text/ photo site pisses me off in general and if I was paying for it? No chance I'd let that fly.
pradn 6y ago
Question about pihole: is it possible to turn off blocking for a website? Do you have to log into the pihole web interface to do that? I often go to websites where some crucial functionality is blocked by my adblocker (ublock origin), where I have to turn it off for that site.
- guilhermetk 6y ago
  Yes, you can do that via whitelist/blacklist: https://docs.pi-hole.net/guides/misc/whitelist-blacklist/ https://docs.pi-hole.net/guides/misc/whitelist-blacklist/
  - secondcoming 6y ago
    allowlist/denylist
    - tsujp 6y ago
      You and everyone else who exhibit this are reading into things that don't exist. Language has context, words are part of language and so therefore words have context too.
      - pkulak 6y ago
        Exactly. And using white/black as synonyms for good/bad may be creating context (connotations, really) that we don't want. It would be fine if we hadn't already overloaded those words to refer to people... but, here we are. In the context we've created. ¯\_(ツ)_/¯
        cgriswald 6y ago
        The original poster used the terms used by the technology. The best choices for changing this terminology would be to write a treatise for HN consumption (to reach the community at large) or to contact the authors of the technology that use this terminology (to fix the origin in this case). Sniping a 'random internet poster' is just lazy trolling.
pkulak 6y ago
I run Pi-Hole on my network as well; it's wonderful. I'm terrified that it will stop working soon though, as companies start to use their own DNS servers, which I've heard is happening.
- JKCalhoun 6y ago
  Interesting. I would think though that the move to their own DNS servers could extend to their own ads as well — that is, cutting out the middle man that is Google/etc. I'm all for news sites, for example, hoisting ads if I knew they were getting the money from those ads, knew the ads were actually coming from their site.
  - pkulak 6y ago
    Sorry, I didn't mean their _own_ servers, I just meant hard-coding 8.8.8.8 into the DNS settings, for example. I wonder if you could hijack those requests at your router and send them back to your Pi-Hole? But then they just switch to DNS over TLS...
    - varenc 6y ago
      I just have my network block outgoing DNS queries that aren’t from the gateway. But you’re so right that DoH is going to throw a wrench in this.
      - deleted 6y ago
        [deleted]
      - letitbeirie 6y ago
        If an ad can use DoH to sidestep a firewall, so can an employee. If Google and Facebook were cunning (and nefarious, but that much is presumed), they would be aggressively developing a product that solves this problem for corporate networks, but at an enormous cost. Otherwise, when corporate networks solve this (and they will), home users who hate ads will just follow whatever pattern they settle on.
JKCalhoun 6y ago
25-33% of requests? Or is this a percentage of bytes? Because I wonder what percentage of bandwidth (in terms of bytes) trackers/banners/ads account for. Need to set up a pi-hole ... just too many other projects....
- cptskippy 6y ago
  Pi-Hole is a DNS solution so it's just blocking DNS lookups. Mine is currently blocking 43.9% of all DNS requests.
- Fiahil 6y ago
  It's a percent of DNS requests. It might be quite difficult to see what percentage of bytes it translate to, since HTTP requests aren't actually sent. My pihole is showing 18.7%-23% of requests blocked :)
- 6gvONxR4sf7o 6y ago
  I set it up recently. It's about as much effort as buying and setting up a new laptop with ubuntu if you get a kit. I'd imagined it as a project beforehand, but in reality it's super easy and trivial (assuming you're comfortable using linux and ssh at a noob level).
  - warent 6y ago
    Yep same. I'm a mere web developer that mostly works on Mac. Getting Pihole setup only took me like an afternoon after having an RPi sitting around doing nothing for months. They make it really easy, just follow the instructions. Also I'm lucky enough that my router has a friendly interface where it's easy to set the router DNS to pihole.
- mstade 6y ago
  How would you measure bytes if the requests are blocked?
  - dylan604 6y ago
    Some people are good at thinking a process through to the end. Others are not and ask questions at the first unknown. It's a large part of why I'm not a teacher.
xeonoex 6y ago
Pi-Hole/NextDNS also blocks adds in most apps. I used NextDNS (which has a limit on the free tier), and recently switched to pihole running on my home server. I also use ZeroTier to connect to my server directly even when I am not on my local network to still use it as the DNS server. Works great.
- StavrosK 6y ago
  I paid them, $20/yr is quite good and I can add my parents' house, my in-laws' house, etc on the same plan and manage them all centrally.
- rndomsrmn 6y ago
  You can get even better coverage with the NoTracking lists (dnsmasq/unbound or dnscrypt-proxy) https://github.com/notracking/hosts-blocklists https://github.com/notracking/hosts-blocklists They focus not only on tracking but also malware prevention, where possible via dns filtering. Pi-Hole still does not properly support wildcard filtering, only via regex but that is not really efficient (requires tons of resources).
dheera 6y ago
I added this to my /etc/hosts https://github.com/StevenBlack/hosts https://github.com/StevenBlack/hosts What is the advantage of having DNS on a separate device other than that it provides ad blocking for multiple devices?
- varenc 6y ago
  That’s the main benefit. But also you can have more flexible block patterns. I run DNSCrypt-Proxy and my block lists can have wildcards. With /etc/hosts you have to enumerate each origin. It can also do things like IP blocking where if any domain resolves to a known ad network IP, then that request is blocked. But mainly, DNSCrypt-proxy encrypts all my outgoing queries and round robins them across resolvers. (Also hi dheera!)
rodgerd 6y ago
Mine was that high, until I ditched all the family Android devices. It's now around 2-3%. It's quite an extraordinary difference.
mstade 6y ago
I use NextDNS for this, it's brilliant. (I'm not affiliated with them in any way, just a happy customer.)
- jeanofthedead 6y ago
  Same. I got tired of Pihole breaking for one reason or another (although I certainly adore the project). NextDNS works extremely well, provides a native app for every device, runs on my router, and is dead simple to maintain.
  - mstade 6y ago
    Hear hear. The only problem I’ve had was when I set it up on my router and my IKEA smart lights stopped responding after some 30 min or so. Turns out the gateway phones home and those calls were blocked, so for some reason or another the gateway just stopped responding to commands. Restarting it or resetting the network made it fly again, but only for the set time before it phoned home again. I was very disappointed by that, after having read some article here on HN arguing that IKEA had actually done IoT sort of right. Oh well. Obviously not a NextDNS specific issue, it’d happen with anything that blocks the call, but just putting it out there for the next sucker that tries to google why their IKEA gateway suddenly stops responding.
karakanb 6y ago
Would you mind sharing the blocklists you use? I have gotten to a ratio like that, but I have noticed that it was causing more issues with regular websites for my guests, so I removed many of the custom ones. I'd like to try some others if you have suggestions.
- warent 6y ago
  [i] Target: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts [] Status: Retrieval successful [i] Received 59896 domains [i] Target: https://mirror1.malwaredomains.com/files/justdomains [] Status: Not found [] List download failed: using previously cached list [i] Received 26854 domains [i] Target: https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt [] Status: No changes detected [i] Received 34 domains [i] Target: https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt [] Status: No changes detected [i] Received 2701 domains [i] Target: https://dbl.oisd.nl/ [] Status: Retrieval successful [i] Received 1167690 domains [i] Target: https://phishing.army/download/phishing_army_blocklist_extended.txt [] Status: Retrieval successful [i] Received 21379 domains [i] Target: https://raw.githubusercontent.com/deathbybandaid/piholeparser/master/Subscribable-Lists/ParsedBlacklists/AakList.txt [] Status: Retrieval successful [i] Received 5 domains [i] Target: https://raw.githubusercontent.com/deathbybandaid/piholeparser/master/Subscribable-Lists/ParsedBlacklists/Prebake-Obtrusive.txt [] Status: Retrieval successful [i] Received 3 domains [i] Target: https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-blocklist.txt [] Status: Retrieval successful [i] Received 14724 domains [i] Target: https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-malware.txt [] Status: Retrieval successful [i] Received 412 domains [i] Target: https://raw.githubusercontent.com/hectorm/hmirror/master/data/adaway.org/list.txt [] Status: Retrieval successful [i] Received 9182 domains [i] Target: https://raw.githubusercontent.com/hectorm/hmirror/master/data/disconnect.me-ad/list.txt [] Status: Retrieval successful [i] Received 2701 domains [i] Target: https://raw.githubusercontent.com/notracking/hosts-blocklists/master/hostnames.txt [] Status: Retrieval successful [i] Received 209608 domains
ycombinete 6y ago
Would Pihole affect latency in online games?
mixmastamyk 6y ago
I use the noscript extension that uses a scripting whitelist. Bit of a pain at first, but pretty soon your browser will be flying. No extra hardware needed. Kid's computer has dnsmasq as a similar solution.
andreasha 6y ago
Noticed that some apps on iOS spammed the DNS server if they couldn't connect to their ad networks which should affect battery negatively.
wejick 6y ago
My issue with pihole or any other DNS adblocker is I can't whitelist some website that I love. As evil as ada network, but I still want my favorite site to get some revenue.
- McDyver 6y ago
  Maybe your favourite site would welcome a direct contribution, instead of an ad click
- deleted 6y ago
  [deleted]

by warent 6y ago

ogre_codes 6y ago

I just have a decent set of ad blockers and the experience is similar. Unfortunately, it often results in weird experiences or I get sites which don't work at all if you have ad blocking.

Average_ 6y ago
Well that's interesting. For me 99.5% of websites work perfectly using ublock origin. The only .5% remaining are websites that actively refuse to serve any kind of adblock users, not because it breaks functionality on their site. I don't think I can recall having visited a single website that would have features break unintentionally because of ublock in the past few years.
- vianneychevalie 6y ago
  Can't agree with you, Dynamics 365 is one of them (it's shit but I've implemented it at work). EDF (French main electricity provider) also breaks for me. That's one example from a big company, and one example with a big user base.
- ev1 6y ago
  It depends on how many privacy lists you have added, probably. Normal display ads all being blocked is generally fine 99% of the time, but if you care about not being permanently tracked across the internet then there are a couple more domains you have to add - except some sites make it mandatory that those invasive fingerprinting scripts and port scanners run and report back a session, otherwise you're refused login or banned.
- ogre_codes 6y ago
  I don't think my experience is vastly different from yours. I do get some sites where pop-overs or cookie notifications are blocked but it's not clear and you just can't scroll. I could turn off those blocker settings, but the notifications are annoying enough it's worth it.
- letitbeirie 6y ago
  The Denver Post just lost my business over this. They have one of those things that scrambles all the words for any user with the audacity to not want to see video+audio ads while reading their newspaper. Is it their content to do what they want with? Sure. Does the same logic apply to the $9 I used to give them each month? You're damn right.
  - ogre_codes 6y ago
    Hard agree. If I'm paying for content, I'd accept a small number of discrete advertising. Video advertising on a text/ photo site pisses me off in general and if I was paying for it? No chance I'd let that fly.

pradn 6y ago

Question about pihole: is it possible to turn off blocking for a website? Do you have to log into the pihole web interface to do that? I often go to websites where some crucial functionality is blocked by my adblocker (ublock origin), where I have to turn it off for that site.

guilhermetk 6y ago
Yes, you can do that via whitelist/blacklist: https://docs.pi-hole.net/guides/misc/whitelist-blacklist/ https://docs.pi-hole.net/guides/misc/whitelist-blacklist/
- secondcoming 6y ago
  allowlist/denylist
  - tsujp 6y ago
    You and everyone else who exhibit this are reading into things that don't exist. Language has context, words are part of language and so therefore words have context too.
    - pkulak 6y ago
      Exactly. And using white/black as synonyms for good/bad may be creating context (connotations, really) that we don't want. It would be fine if we hadn't already overloaded those words to refer to people... but, here we are. In the context we've created. ¯\_(ツ)_/¯
      - cgriswald 6y ago
        The original poster used the terms used by the technology. The best choices for changing this terminology would be to write a treatise for HN consumption (to reach the community at large) or to contact the authors of the technology that use this terminology (to fix the origin in this case). Sniping a 'random internet poster' is just lazy trolling.

pkulak 6y ago

I run Pi-Hole on my network as well; it's wonderful. I'm terrified that it will stop working soon though, as companies start to use their own DNS servers, which I've heard is happening.

JKCalhoun 6y ago
Interesting. I would think though that the move to their own DNS servers could extend to their own ads as well — that is, cutting out the middle man that is Google/etc. I'm all for news sites, for example, hoisting ads if I knew they were getting the money from those ads, knew the ads were actually coming from their site.
- pkulak 6y ago
  Sorry, I didn't mean their _own_ servers, I just meant hard-coding 8.8.8.8 into the DNS settings, for example. I wonder if you could hijack those requests at your router and send them back to your Pi-Hole? But then they just switch to DNS over TLS...
  - varenc 6y ago
    I just have my network block outgoing DNS queries that aren’t from the gateway. But you’re so right that DoH is going to throw a wrench in this.
    - deleted 6y ago
      [deleted]
    - letitbeirie 6y ago
      If an ad can use DoH to sidestep a firewall, so can an employee. If Google and Facebook were cunning (and nefarious, but that much is presumed), they would be aggressively developing a product that solves this problem for corporate networks, but at an enormous cost. Otherwise, when corporate networks solve this (and they will), home users who hate ads will just follow whatever pattern they settle on.

JKCalhoun 6y ago

25-33% of requests? Or is this a percentage of bytes? Because I wonder what percentage of bandwidth (in terms of bytes) trackers/banners/ads account for. Need to set up a pi-hole ... just too many other projects....

cptskippy 6y ago
Pi-Hole is a DNS solution so it's just blocking DNS lookups. Mine is currently blocking 43.9% of all DNS requests.
Fiahil 6y ago
It's a percent of DNS requests. It might be quite difficult to see what percentage of bytes it translate to, since HTTP requests aren't actually sent. My pihole is showing 18.7%-23% of requests blocked :)
6gvONxR4sf7o 6y ago
I set it up recently. It's about as much effort as buying and setting up a new laptop with ubuntu if you get a kit. I'd imagined it as a project beforehand, but in reality it's super easy and trivial (assuming you're comfortable using linux and ssh at a noob level).
- warent 6y ago
  Yep same. I'm a mere web developer that mostly works on Mac. Getting Pihole setup only took me like an afternoon after having an RPi sitting around doing nothing for months. They make it really easy, just follow the instructions. Also I'm lucky enough that my router has a friendly interface where it's easy to set the router DNS to pihole.
mstade 6y ago
How would you measure bytes if the requests are blocked?
- dylan604 6y ago
  Some people are good at thinking a process through to the end. Others are not and ask questions at the first unknown. It's a large part of why I'm not a teacher.

xeonoex 6y ago

Pi-Hole/NextDNS also blocks adds in most apps. I used NextDNS (which has a limit on the free tier), and recently switched to pihole running on my home server. I also use ZeroTier to connect to my server directly even when I am not on my local network to still use it as the DNS server. Works great.

StavrosK 6y ago
I paid them, $20/yr is quite good and I can add my parents' house, my in-laws' house, etc on the same plan and manage them all centrally.
rndomsrmn 6y ago
You can get even better coverage with the NoTracking lists (dnsmasq/unbound or dnscrypt-proxy) https://github.com/notracking/hosts-blocklists https://github.com/notracking/hosts-blocklists They focus not only on tracking but also malware prevention, where possible via dns filtering. Pi-Hole still does not properly support wildcard filtering, only via regex but that is not really efficient (requires tons of resources).

dheera 6y ago

I added this to my /etc/hosts https://github.com/StevenBlack/hosts https://github.com/StevenBlack/hosts What is the advantage of having DNS on a separate device other than that it provides ad blocking for multiple devices?

varenc 6y ago
That’s the main benefit. But also you can have more flexible block patterns. I run DNSCrypt-Proxy and my block lists can have wildcards. With /etc/hosts you have to enumerate each origin. It can also do things like IP blocking where if any domain resolves to a known ad network IP, then that request is blocked. But mainly, DNSCrypt-proxy encrypts all my outgoing queries and round robins them across resolvers. (Also hi dheera!)

rodgerd 6y ago

Mine was that high, until I ditched all the family Android devices. It's now around 2-3%. It's quite an extraordinary difference.

mstade 6y ago

I use NextDNS for this, it's brilliant. (I'm not affiliated with them in any way, just a happy customer.)

jeanofthedead 6y ago
Same. I got tired of Pihole breaking for one reason or another (although I certainly adore the project). NextDNS works extremely well, provides a native app for every device, runs on my router, and is dead simple to maintain.
- mstade 6y ago
  Hear hear. The only problem I’ve had was when I set it up on my router and my IKEA smart lights stopped responding after some 30 min or so. Turns out the gateway phones home and those calls were blocked, so for some reason or another the gateway just stopped responding to commands. Restarting it or resetting the network made it fly again, but only for the set time before it phoned home again. I was very disappointed by that, after having read some article here on HN arguing that IKEA had actually done IoT sort of right. Oh well. Obviously not a NextDNS specific issue, it’d happen with anything that blocks the call, but just putting it out there for the next sucker that tries to google why their IKEA gateway suddenly stops responding.

karakanb 6y ago

Would you mind sharing the blocklists you use? I have gotten to a ratio like that, but I have noticed that it was causing more issues with regular websites for my guests, so I removed many of the custom ones. I'd like to try some others if you have suggestions.

warent 6y ago
[i] Target: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts [] Status: Retrieval successful [i] Received 59896 domains [i] Target: https://mirror1.malwaredomains.com/files/justdomains [] Status: Not found [] List download failed: using previously cached list [i] Received 26854 domains [i] Target: https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt [] Status: No changes detected [i] Received 34 domains [i] Target: https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt [] Status: No changes detected [i] Received 2701 domains [i] Target: https://dbl.oisd.nl/ [] Status: Retrieval successful [i] Received 1167690 domains [i] Target: https://phishing.army/download/phishing_army_blocklist_extended.txt [] Status: Retrieval successful [i] Received 21379 domains [i] Target: https://raw.githubusercontent.com/deathbybandaid/piholeparser/master/Subscribable-Lists/ParsedBlacklists/AakList.txt [] Status: Retrieval successful [i] Received 5 domains [i] Target: https://raw.githubusercontent.com/deathbybandaid/piholeparser/master/Subscribable-Lists/ParsedBlacklists/Prebake-Obtrusive.txt [] Status: Retrieval successful [i] Received 3 domains [i] Target: https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-blocklist.txt [] Status: Retrieval successful [i] Received 14724 domains [i] Target: https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-malware.txt [] Status: Retrieval successful [i] Received 412 domains [i] Target: https://raw.githubusercontent.com/hectorm/hmirror/master/data/adaway.org/list.txt [] Status: Retrieval successful [i] Received 9182 domains [i] Target: https://raw.githubusercontent.com/hectorm/hmirror/master/data/disconnect.me-ad/list.txt [] Status: Retrieval successful [i] Received 2701 domains [i] Target: https://raw.githubusercontent.com/notracking/hosts-blocklists/master/hostnames.txt [] Status: Retrieval successful [i] Received 209608 domains

ycombinete 6y ago

Would Pihole affect latency in online games?

mixmastamyk 6y ago

I use the noscript extension that uses a scripting whitelist. Bit of a pain at first, but pretty soon your browser will be flying. No extra hardware needed. Kid's computer has dnsmasq as a similar solution.

andreasha 6y ago

Noticed that some apps on iOS spammed the DNS server if they couldn't connect to their ad networks which should affect battery negatively.

wejick 6y ago

My issue with pihole or any other DNS adblocker is I can't whitelist some website that I love. As evil as ada network, but I still want my favorite site to get some revenue.

McDyver 6y ago
Maybe your favourite site would welcome a direct contribution, instead of an ad click
deleted 6y ago
[deleted]