4 ms·

A related question for HN... If I want a network-wide ad blocking device that understands wildcard host names that's cheap & low-wattage, is a Raspberry Pi wit

by jasode 9y ago

A related question for HN... If I want a network-wide ad blocking device that understands wildcard host names that's cheap & low-wattage, is a Raspberry Pi with Pi-hole[1] the best option? The PAC proxy autoconfig in this article is fine for configuring individual browsers (especially when browsing on iPad/iPhone away from home such as airports) but I want a site-wide firewall at the DNS network level to block: - IoT like LG/Samsung tvs sending privacy data - MS Windows 10 telemetry data - smartphones/tablets ads where the proxy wasn't set - ...everything else I didn't think about [1] https://pi-hole.net/ https://pi-hole.net/

olympus 9y ago
Best, as in cheapest, or highest throughput, or easiest to configure? Your network type matters as well. A pi-hole works okay and is a low price option for a home network, but would get swamped on a work network. My suggestion: make a pi-hole, and see what you like about it. If it doesn't work for you, buy a real firewall from Cisco or similar. Then use the raspberry pi as a media machine or some other project.
hnlmorg 9y ago
AFAIK dnsmasq (which Pi-hole uses) doesn't support wildcards. However Pi-hole does import hugely extensive lists of domains so you'll unlikely to need wildcard matching anyway. Pi-hole is just dnsmasq plus a whole bunch of 3rd party curated lists of bad domains and some helper scripts. It's obviously the helper scripts that give Pi-hole its value but equally it's very easy to build your own equivalent. Which is useful if you think you need something beefier to host it than a Raspberry Pi. Disclaimer: I have built my own Pi-hole equivalent - a few years before Pi-hole was first released in fact. As for you questions about which content gets blocked, my server has successfully been blocking the stuff you described (see caveat below). In fact it was so successful at blocking IoT telemetry that it highlighted a weird bug on Sky+ HD set top boxes where they refused to boot if they couldn't phone home when connected over ethernet. Caveat: I don't run Windows so cannot comment about the Win 10 telemetry data.
- jasode 9y ago
  >AFAIK dnsmasq (which Pi-hole uses) doesn't support wildcards. There seems to be conflicting information about wildcards but I thought the feature was added this year: https://github.com/pi-hole/pi-hole/pull/1065 https://github.com/pi-hole/pi-hole/pull/1065 >However Pi-hole does import hugely extensive lists of domains so you'll unlikely to need wildcard matching anyway. I liked the power of wildcards because the ad and malware networks always add new subdomains faster than the updating and propagating of static hosts lists. Example might be: 0.0.0.0 ads.zedo.com 0.0.0.0 c1.zedo.com 0.0.0.0 c2.zedo.com 0.0.0.0 c3.zedo.com 0.0.0.0 c4.zedo.com 0.0.0.0 c5.zedo.com 0.0.0.0 c6.zedo.com 0.0.0.0 c7.zedo.com 0.0.0.0 c8.zedo.com 0.0.0.0 d2.zedo.com 0.0.0.0 d3.zedo.com If zedo.com adds new subdomains "c9.zedo.com" and "d4.zedo.com", the static list lets the ads through until I update the lists. Wildcards such as "* .zedo.com" are an elegant syntax to stop that loophole. >cannot comment about the Win 10 telemetry data. Btw, I mentioned MS Win 10 because Microsoft added code at the kernel level that ignores manual entries edded to /etc/hosts file so it can still phone home. Therefore, only a firewall device that lives outside of Win 10 can block it. (https://www.petri.com/windows-10-ignoring-hosts-file-specific-name-resolution https://www.petri.com/windows-10-ignoring-hosts-file-specifi...)
  - hnlmorg 9y ago
    Ahh cool. Wasn't aware you could do wildcards. I might have to have another play with dnsmasq tonight :)
Fnoord 9y ago
Last time I looked into Pi-Hole, it worked via DNS. though perhaps they added options here (like with a proxy, as you mentioned). I'm not sure if PAC can still be MITMed. Old iOS devices can't install OS-wide adblocker (these go for a few USD in the App Store otherwise, and work OS-wide). For them, Pi-Hole is a nice solution. A better solution is to replace the entire devices though. You can fix the issue also by just not using smart TVs, and by fixing MS Windows 10 manually with a tool like DWS [1]. For a mobile phone, Pi-Hole works whilst being in the (W)LAN. If you want to have it on the go as well you're going to have to use something like a VPN, have an always-on, low latency connection at home (e.g. fiber, or recent cable/DSL technology, otherwise an option could be cloud). This is going to have small overhead on data; not much to affect your plan in a great way. The problem is your phone battery! Imagine you have a push connection up on WhatsApp. Now it also needs to reconnect to the VPN, and encrypt the data twice: once E2E with WhatsApp and once between you and the VPN. With no additional security for WhatsApp. Some mobile phone carriers restrict VPN data, or mangle DNS data. They use a transparent proxy to route DNS traffic via their DNS server, so its pointless to have your Pi-Hole listen on your WAN (besides, 3rd party would be able to use your Pi-Hole as well). You want to have your DNS traffic through your VPN instead. However, once again, that's additional overhead. The advantage though is that your DNS traffic cannot be MITMed whilst e.g. being in a coffeeshop. What I do is, I just buy the app or buy a subscription on apps I use a lot, and remove apps with ads which I don't use a lot. I also have targeted ads off in Google settings, so I never get relevant ads either way. I don't want to see relevant ads -if I see ads- because I don't want to limit myself being affected by advertising. Such limiting is always relative; never absolute. Advertising works in many sneaky ways, including e.g. paid advertisement on a popular YouTube vlog channel). Furthermore, uBlock Origin has low impact on phone performance compared to DNS-based blocking. It is better than nothing. Using NoScript can also greatly help, esp if you don't have 4G or have a slower smartphone. [1] https://github.com/Nummer/Destroy-Windows-10-Spying https://github.com/Nummer/Destroy-Windows-10-Spying
stanlarroque 9y ago
Dnsmasq with a black list is very powerful for that matter. AlgoVPN [1] is using it, and provides a network-wide ad blocking and anti-tracking. Also, I made an app which is a distributed version of AlgoVPN, for iOS devices. People testing it find it very convenient. [1] https://github.com/trailofbits/algo https://github.com/trailofbits/algo
cbr 9y ago
hostname-based blocking can't handle when ads are served from the same domain as content you do want
joosters 9y ago
Privoxy - https://www.privoxy.org/ https://www.privoxy.org/ is a light-weight HTTP and HTTPs proxy that can be used as a network-wide ad blocking service, and is small and efficient enough to run fine on a Pi (it's available packaged with most linux distributions) For HTTP requests, you can configure lists of URLs (or URL fragments) to block, allowing for full ad blocking. It can also do tricks like allowing URLs but blocking or discarding their cookies so the sites are usable but can't track you via their cookies. For HTTPs requests, the proxy can't see the URLs so it can't block based on URL, but it falls back to blocking based on hostname, which is good enough for most situations. Browser plugins like 'Proxy SwitchyOmega' (Firefox/Chrome) make it easy to switch between using the proxy or not if you really need to skip the blocking. It's also possible to set up privoxy as a transparent proxy, meaning that everything on your network is forced to use it to communicate with the outside world, so it can work with any device, not just those that have proxy configuration options). I definitely recommend it, it runs fine on my home network. My TV and PS4 work well with it, for instance. (it's a shame that I have to resort to ad-blocking my television in order to stop it spying on me...)
- jdironman 9y ago
  What about solutions for larger environments such as education / universities with 500+ users?
  - AdieuToLogic 9y ago
    Squid[0] might be a good fit for those types of environments. 0 - http://www.squid-cache.org/ http://www.squid-cache.org/
  - ktta 9y ago
    You are getting into dangerous territory if you want to MITM so many HTTPS connections. (You can do anything with HTTP you want. Starting with DNS blackholing is best) Note that the software GP linked to doesn't (can't) strip ads off an HTTPS connection[1]. It can only completely block it. So for ads + content from a single origin, you can't do anything. If you really want get into individual HTTPS connections, you'd have to install a root cert into every single once of the computers and MITM them by presenting your cert. This is only done in scenarios where one has to control everything the client machines do. Otherwise you're (much) better off preinstalling uBlock Origin on all the computers. BlueCoat is a company which works in this space[2] (it acquired by Symantec last year). Even then, you risk breaking something sometime[3] [1]: https://www.privoxy.org/faq/misc.html#SSL https://www.privoxy.org/faq/misc.html#SSL [2]: https://www.bluecoat.com/support-services https://www.bluecoat.com/support-services [3]: https://news.ycombinator.com/item?id=13750379 https://news.ycombinator.com/item?id=13750379
tinus_hn 9y ago
> I want a site-wide firewall at the DNS network level to block ... What does that mean? Seems like you want a device that blocks ‘everything bad’ and lets through ‘everything good’. Unfortunately it’s not possible to specify exactly what’s good and bad, especially not in a general way.

by jasode 9y ago

olympus 9y ago

Best, as in cheapest, or highest throughput, or easiest to configure? Your network type matters as well. A pi-hole works okay and is a low price option for a home network, but would get swamped on a work network. My suggestion: make a pi-hole, and see what you like about it. If it doesn't work for you, buy a real firewall from Cisco or similar. Then use the raspberry pi as a media machine or some other project.

hnlmorg 9y ago

AFAIK dnsmasq (which Pi-hole uses) doesn't support wildcards. However Pi-hole does import hugely extensive lists of domains so you'll unlikely to need wildcard matching anyway. Pi-hole is just dnsmasq plus a whole bunch of 3rd party curated lists of bad domains and some helper scripts. It's obviously the helper scripts that give Pi-hole its value but equally it's very easy to build your own equivalent. Which is useful if you think you need something beefier to host it than a Raspberry Pi. Disclaimer: I have built my own Pi-hole equivalent - a few years before Pi-hole was first released in fact. As for you questions about which content gets blocked, my server has successfully been blocking the stuff you described (see caveat below). In fact it was so successful at blocking IoT telemetry that it highlighted a weird bug on Sky+ HD set top boxes where they refused to boot if they couldn't phone home when connected over ethernet. Caveat: I don't run Windows so cannot comment about the Win 10 telemetry data.

jasode 9y ago
>AFAIK dnsmasq (which Pi-hole uses) doesn't support wildcards. There seems to be conflicting information about wildcards but I thought the feature was added this year: https://github.com/pi-hole/pi-hole/pull/1065 https://github.com/pi-hole/pi-hole/pull/1065 >However Pi-hole does import hugely extensive lists of domains so you'll unlikely to need wildcard matching anyway. I liked the power of wildcards because the ad and malware networks always add new subdomains faster than the updating and propagating of static hosts lists. Example might be: 0.0.0.0 ads.zedo.com 0.0.0.0 c1.zedo.com 0.0.0.0 c2.zedo.com 0.0.0.0 c3.zedo.com 0.0.0.0 c4.zedo.com 0.0.0.0 c5.zedo.com 0.0.0.0 c6.zedo.com 0.0.0.0 c7.zedo.com 0.0.0.0 c8.zedo.com 0.0.0.0 d2.zedo.com 0.0.0.0 d3.zedo.com If zedo.com adds new subdomains "c9.zedo.com" and "d4.zedo.com", the static list lets the ads through until I update the lists. Wildcards such as "* .zedo.com" are an elegant syntax to stop that loophole. >cannot comment about the Win 10 telemetry data. Btw, I mentioned MS Win 10 because Microsoft added code at the kernel level that ignores manual entries edded to /etc/hosts file so it can still phone home. Therefore, only a firewall device that lives outside of Win 10 can block it. (https://www.petri.com/windows-10-ignoring-hosts-file-specific-name-resolution https://www.petri.com/windows-10-ignoring-hosts-file-specifi...)
- hnlmorg 9y ago
  Ahh cool. Wasn't aware you could do wildcards. I might have to have another play with dnsmasq tonight :)

Fnoord 9y ago

Last time I looked into Pi-Hole, it worked via DNS. though perhaps they added options here (like with a proxy, as you mentioned). I'm not sure if PAC can still be MITMed. Old iOS devices can't install OS-wide adblocker (these go for a few USD in the App Store otherwise, and work OS-wide). For them, Pi-Hole is a nice solution. A better solution is to replace the entire devices though. You can fix the issue also by just not using smart TVs, and by fixing MS Windows 10 manually with a tool like DWS [1]. For a mobile phone, Pi-Hole works whilst being in the (W)LAN. If you want to have it on the go as well you're going to have to use something like a VPN, have an always-on, low latency connection at home (e.g. fiber, or recent cable/DSL technology, otherwise an option could be cloud). This is going to have small overhead on data; not much to affect your plan in a great way. The problem is your phone battery! Imagine you have a push connection up on WhatsApp. Now it also needs to reconnect to the VPN, and encrypt the data twice: once E2E with WhatsApp and once between you and the VPN. With no additional security for WhatsApp. Some mobile phone carriers restrict VPN data, or mangle DNS data. They use a transparent proxy to route DNS traffic via their DNS server, so its pointless to have your Pi-Hole listen on your WAN (besides, 3rd party would be able to use your Pi-Hole as well). You want to have your DNS traffic through your VPN instead. However, once again, that's additional overhead. The advantage though is that your DNS traffic cannot be MITMed whilst e.g. being in a coffeeshop. What I do is, I just buy the app or buy a subscription on apps I use a lot, and remove apps with ads which I don't use a lot. I also have targeted ads off in Google settings, so I never get relevant ads either way. I don't want to see relevant ads -if I see ads- because I don't want to limit myself being affected by advertising. Such limiting is always relative; never absolute. Advertising works in many sneaky ways, including e.g. paid advertisement on a popular YouTube vlog channel). Furthermore, uBlock Origin has low impact on phone performance compared to DNS-based blocking. It is better than nothing. Using NoScript can also greatly help, esp if you don't have 4G or have a slower smartphone. [1] https://github.com/Nummer/Destroy-Windows-10-Spying https://github.com/Nummer/Destroy-Windows-10-Spying

stanlarroque 9y ago

Dnsmasq with a black list is very powerful for that matter. AlgoVPN [1] is using it, and provides a network-wide ad blocking and anti-tracking. Also, I made an app which is a distributed version of AlgoVPN, for iOS devices. People testing it find it very convenient. [1] https://github.com/trailofbits/algo https://github.com/trailofbits/algo

cbr 9y ago

hostname-based blocking can't handle when ads are served from the same domain as content you do want

joosters 9y ago

Privoxy - https://www.privoxy.org/ https://www.privoxy.org/ is a light-weight HTTP and HTTPs proxy that can be used as a network-wide ad blocking service, and is small and efficient enough to run fine on a Pi (it's available packaged with most linux distributions) For HTTP requests, you can configure lists of URLs (or URL fragments) to block, allowing for full ad blocking. It can also do tricks like allowing URLs but blocking or discarding their cookies so the sites are usable but can't track you via their cookies. For HTTPs requests, the proxy can't see the URLs so it can't block based on URL, but it falls back to blocking based on hostname, which is good enough for most situations. Browser plugins like 'Proxy SwitchyOmega' (Firefox/Chrome) make it easy to switch between using the proxy or not if you really need to skip the blocking. It's also possible to set up privoxy as a transparent proxy, meaning that everything on your network is forced to use it to communicate with the outside world, so it can work with any device, not just those that have proxy configuration options). I definitely recommend it, it runs fine on my home network. My TV and PS4 work well with it, for instance. (it's a shame that I have to resort to ad-blocking my television in order to stop it spying on me...)

jdironman 9y ago
What about solutions for larger environments such as education / universities with 500+ users?
- AdieuToLogic 9y ago
  Squid[0] might be a good fit for those types of environments. 0 - http://www.squid-cache.org/ http://www.squid-cache.org/
- ktta 9y ago
  You are getting into dangerous territory if you want to MITM so many HTTPS connections. (You can do anything with HTTP you want. Starting with DNS blackholing is best) Note that the software GP linked to doesn't (can't) strip ads off an HTTPS connection[1]. It can only completely block it. So for ads + content from a single origin, you can't do anything. If you really want get into individual HTTPS connections, you'd have to install a root cert into every single once of the computers and MITM them by presenting your cert. This is only done in scenarios where one has to control everything the client machines do. Otherwise you're (much) better off preinstalling uBlock Origin on all the computers. BlueCoat is a company which works in this space[2] (it acquired by Symantec last year). Even then, you risk breaking something sometime[3] [1]: https://www.privoxy.org/faq/misc.html#SSL https://www.privoxy.org/faq/misc.html#SSL [2]: https://www.bluecoat.com/support-services https://www.bluecoat.com/support-services [3]: https://news.ycombinator.com/item?id=13750379 https://news.ycombinator.com/item?id=13750379

tinus_hn 9y ago

> I want a site-wide firewall at the DNS network level to block ... What does that mean? Seems like you want a device that blocks ‘everything bad’ and lets through ‘everything good’. Unfortunately it’s not possible to specify exactly what’s good and bad, especially not in a general way.