23 ms·

How I re–over-engineered my home network for privacy and security

345 points by chmaynard 5y ago

binarybanana 5y ago
For anyone who wants to go a similar route I can highly recommend OpenWrt as the OS. Not only does it run on cheap routers, but also on big x86 machines and even in VMs and/or containers. For example, I run OpenWrt on my desktop in a LXC container to manage networking (Wifi, bridging firewall for bridging VMs into the network, general firewall, etc.) through the nice webui. It gets direct to access the WiFi adapter and the host gets access through a bridge connected to the container via a VETH interface. It comes with nice addon packages for stuff like Wireguard, all kinds of tunnels/VPN, adblockers, runs containers and a ton more. I even run it on a VPS as container with it having exclusive access to the "physical" NIC. The parent OS isn't directly accessible at all. Makes firewalling a breeze. The only open ports are for the Tor relay and Wireguard, through Wich I connect to the webui/ssh and do everything else. Of course, my router also runs OpenWrt...
- atatatat 5y ago
  > big x86 machines You saw the "security" in the title, right?
- carlhjerpe 5y ago
  How is the OpenWrt CLI? Say I want to deploy it with Ansible. How does it stack up against VyOS (which just recently got VRF-lite support)
  - dtx1 5y ago
    > How is the OpenWrt CLI You SSH into it. Then do whatever you need to. There's nothing in the UI that can't be done via CLI as far as I know. Some plugins might not be 100% CLI compliant but at least the Base UI (luci) is completely transparent to the CLI via uci.
    - HighPlainsDrftr 5y ago
      Do you have any recommended hardware for OpenWRT? I've been wanting to put in a low powered router/firewall on my home network that isn't controlled by a big vendor. I haven't done a ton of research in this area, but I'd certainly like to use OpenWRT or OPNSense on my home routers/firewalls. Side note: I've been trying to figure out a decent way to get rid of Android on my Galaxy S9+, but it appears to be locked. At the end of the day, I just want to be in control of my bandwidth, my data, and know whats going on. Big companies are making this very complicated with all the tracking. I recently re-enabled my pi-hole on a virtual machine, and it never ceases to amaze me what is talking to the internet without my permission. After digging into DoH a bit, I'm about to the point where I think I need to put in an outbound proxy, deny all outbound access except via the proxy, and iterate again and again. I just don't want to have $200 a month in power bills to support my home network to save bandwidth and know what is traversing the net.
      - dtx1 5y ago
        WRT3200ACM is what I use. It's one of the most powerful consumer devices with wifi supported by openwrt. It's powerful enough for a router with a little firewalling, VPN (wireguard only if you want speed), DoH and some Ad Filtering but thats pushing it's limits from my experience. If you want more power https://openbsdrouterguide.net/ https://openbsdrouterguide.net/ is your friend. That being said there is no hardcore prosumer hardware out there for this purpose. The moment you go beyond home user router hardware like the WRT3200acm you are in either CISCO Buisness stuff or custom server builds. Potentially a Raspberry Pi 4 with a PCIE ethernet card is closest to prosumer hardware out there and there's a lot of hacking involved to get that running to the same degree as a openwrt router
        HighPlainsDrftr 5y ago
        This is interesting. I'm going to research it further. I really appreciate the feedback. I'm really starting to hate DoH - and I may just not put any IoT thing on my network that uses it. Maybe that's the way to go. But I doubt most consumers really care. It's complicated.
  - rhn_mk1 5y ago
    OpenWRT doesn't pack Python out of the box, so you need to install it, and have enough space for it, if you want to use Ansible. It uses a custom stack for configuration (like VyOS), so builtin ansible tasks won't always be so helpful. Configuration is not stored in a single place, but in several files, and it's easy to lock yourself out while testing changes: there's no commit-timeout, and there's no committing of changes, nor rollback. It's just editing random files, and restarting services. I think there's a special configuration command that might fix some of the above issues, but I've been using the web interface (which actually does support committing and, to some extent, validation).
- OrvalWintermute 5y ago
  I thought about which OS for some of the same things and I realized that I would rather go with a lab version of a full enterprise firewall. A Palo Alto VM gets you pretty much most of the sweet PA features without the cost, and a better approach than an outdated strategy like VLAN as Access Control, or zone firewalling, permitting the use of permit/deny by protocol, and overall better privilege tiering by network area.
  - EvanAnderson 5y ago
    > A Palo Alto VM ... without the cost ... Does Palo Alto have some kind of no-cost offering in their VM line?
    - OrvalWintermute 5y ago
      > A Palo Alto VM ... without the cost ... This isn't free, but $50-200 is alot less than $2-4k
      - EvanAnderson 5y ago
        Yikes! I didn't realize PA had anything available for less than a 4 digit price point. I'll check it out. Thanks!
  - todd8 5y ago
    I’m curious about PA firewalls. The product descriptions claim “Machine Learning” based routing/firewalls. What does that even mean? I’m a bit skeptical about AI being used in a firewall. Can someone help me understand why I should consider this instead of running pfsense on a Netgate appliance?
- eric__cartman 5y ago
  I always found it baffling how a community maintained project like OpenWrt can not only keep routers updated for many years (unlike the manufacturer that gives you an update once in a blue moon for the first 2 years) but make these cheap routers so useful and so stable you just forget the thing is there in the first place. I can't remember the last time I had to reboot a router running OpenWrt because it started behaving erratically. You can do a lot with these commodity ARM CPUs, 64-128MB of ram and a few tens of megabytes of flash storage.
  - maerF0x0 5y ago
    i literally have my comcast router on a 24 hour time and it shutsdown for an hour each night from 3-4am . Communism style brownouts of internet over night
- 123pie123 5y ago
  for home I can recommend openwrt running on a BT Home Hub 5A you can buy them in the uk/ ebay for 15-20 pounds already with openwrt installed (you can do it yourself, but it includes a bit of soldering) - I have two in case the main one fails - talks to most if not all ISPs I love openwrt now, it does take a bit of getting used to if you havent used it before. I mainly use to lock my wifi down between hours for the kids. whilst keeping another wifi/ SSID open. for security all my NAS's are wired and locked down to key wired computers - I keep meaning to create a Nextcloud gateway on docker
- stjohnswarts 5y ago
  I have dd-wrt on a tp-link router and it works great and has been for about 3 years now. No issues. I just have to check the site about once a month and see if there's an update. I wish that was automated or had a notice in the web gui a new one is available. Very configurable and stable.
- 3np 5y ago
  You know, I used to do this and give the same advice as well but after a lot of time spent on it, I am not anymore. The way OpenWRT handles routing and firewall rules is particular and they apply their own terminology for some things. They have their own distro-specific packages for things like DHCP (odhcp(c)d) and firewall (fw3). For very simple networks, it's very smooth to get to where you want. Add on dual-stack v4/v6, vlans, multiple firewall zones, routing policies etc and things start becoming very unpredictable. Oh, and that adblock package? Turns out a single invalid line in a blocklist will completely break DNS (at least on the version I was running from last year). Not to mention that (AFAIK) there's no good way to keep up to date with security patches and bugfixes while keeping the system stable. After all the countless hours I poured into OpenWRT configuration, I finally realized that it's so much less pain and confusion with vanilla Debian with systemd-networkd (which BTW natively supports setting up Wireguard interfaces now) and firewalld+nftables, everything configured via ansible playbooks. For someone diving into this today, it's a lot easier and more future-proof with nftables than iptables - and OpenWRT will be married to iptables for the foreseeable future. It's great that it works for you, but if you like I did have some imposter syndrome over not perfectly understanding Linux networking and are happy that OpenWRT takes care of those confusing iptables rules and routing policies and what-not - you may just discover that learning how it actually works will take less work than abusing OpenWRT into doing what you want. Sure, you have to give up the WebUI and some of the custom add-ons. I am sure BSD or Rocky Linux are fine choices as well; Debian just happens to be what I mostly use for servers otherwise. I don't want to hate too much on OpenWRT as it's great for novices with trivial needs and there are many devices where it or dd-wrt are the only readily available options. But if you run Linux anyway and have an x86/amd64/arm device you're going to use as a main router, I'd recommend choosing a "normal" distro and setting things up from scratch.
syntaxing 5y ago
Some of the authors concerns with Pihole can probably be fixed by replacing it with Adguard home because of the HTTPS support. That being said, it’s kind of cool to see someone higher on the corporate ladder posting these kind of blog posts!
nickphx 5y ago
I would hardly consider blindly trusting a third party like cloudflare to be over-engineering.
- traceroute66 5y ago
  "I over-engineered my network for privacy" .... by trusting a third-party for DNS resolution. Doesn't compute ! Even more so as the OP chose Cloudflare ! US Jursisdiction + Commercial Company ?
  - lostlogin 5y ago
    Have you got a recommendation? I see them as a good choice compared to my other options. but if there is better out there…
    - ignoramous 5y ago
      Use DNSCrypt with anonymized relays: https://github.com/DNSCrypt/dnscrypt-proxy/wiki/Anonymized-DNS https://github.com/DNSCrypt/dnscrypt-proxy/wiki/Anonymized-D... Or, wait until ODoH goes mainstream: https://datatracker.ietf.org/doc/html/draft-pauly-dprive-oblivious-doh-06 https://datatracker.ietf.org/doc/html/draft-pauly-dprive-obl...
    - traceroute66 5y ago
      > Have you got a recommendation? Well, if you're going to go to the trouble of building and running your own VMs or docker images then the obvious recommendation is to run your own recursive resolver (Unbound or Knot Resolver). The added bonus is that CDN content will operate as expected for your IP range instead of you talking to the CDN server nearest to whatever datacentre you spoke to the third-party DNS BGP anycast on. This may or may not be a big deal depending on your geographic location and internet connection. I am aware some third-party DNS services have options to try to fix the CDN problem, but YMMV as to whether this works in practice. Otherwise, if I was forced to name a third-party, I would likely name Quad9. Especially after their recent relocation to Swiss jurisdiction[1]. IIRC (its been a while since I read it) their privacy policy is also better than Cloudflare's too (e.g. no weird, "we'll retain some stuff for 25 hours" IIRC). [1] https://www.quad9.net/news/blog/quad9-public-domain-name-service-moves-to-switzerland-for-maximum-internet-privacy-protection/ https://www.quad9.net/news/blog/quad9-public-domain-name-ser...
late_groomer 5y ago
Really glad to see this write-up. I love my current PiHole setup, but replicating it for family/friends (especially with my blocking so many sites they might want) hasn't seemed doable. I would be having to help them constantly with blocked sites, etc.,. My setup could be even better if either Fios or Orbi provided a half-decent router, but just using PiHole as my DNS server has been awesome. When my Pi (SD card) crashed and I had to revert to my old router setup, it was shocking how slow everything was. I'd become so used to pages loading in a blink, waiting a second or two while ads loaded seemed to take forever. If you have any unused SFF computer gathering dust, give PiHole a try (runs on any of the major linux distros). Initial setup takes under an hour. You can customize the backend out of it with blocksite lists; I'm blocking so much my wife and kids can't even install new iPhone apps. Amazon Devices, Rokus, Google, nothing gets to call home unless I allow it. I have it forcing all Google/Bing searches to use their clean filter; worrying my kids might google "porn" is a thing of the past. There are easy to find tutorials out there, just be sure to filter Google images, as well.
- dreyfan 5y ago
  That sounds overbearing and your mobile data bills will skyrocket as nobody wants to use your annoying home setup.
  - lostlogin 5y ago
    I’ve had more than one person ask how to replicate mine after the standard internet noise was removed while on my wifi. It’s also fairly easy to keep a VPN to home running to avoid nosy cooperate wifi and enjoy my own vicious filters.
- dawson 5y ago
  Did you consider using NextDNS?
- creeble 5y ago
  FWIW, a plug here for The FreshTomato router firmware. I just retired my PiHole because FT does the same job seemingly just as well. Tomato has long been my chosen router firmware - much easier to use than OpenWrt, but not quite as feature-laden and runs on a more select set of hardware (mostly ASUS routers). But it has recently gotten a development boost from a new maintainer, and the ad-blocking seems just as good as PiHole. I suppose I can unplug my PiHole now that there’s no traffic going to it...
ctur 5y ago
This is a great write-up, not just for the ostensible goal (more privacy and security with maintainable infrastructure) but also because it lays out some reusable primitive for other “home lab” type projects. The next stop down this fun slippery slope is kubernetes :) That said, it also is worth mentioning NextDNS as an option. It basically is “pihole as a service” and is reasonably affordable. I ended up going that route to achieve similar goals as to the author and have been happy with it.
- ignoramous 5y ago
  > That said, it also is worth mentioning NextDNS as an option. There's ControlD.com as well (by Windscribe) that's super neat which I really like, that offers more than just DNS. Disclosure: I co-develop a FOSS NextDNS alternative.
- wintermutestwin 5y ago
  Doesn't NextDNS have the same problem as VPN where you are trusting all your browsing data to one entity?
- aheckler 5y ago
  I asked the author of the OP about NextDNS on Twitter. His reply: https://twitter.com/benbalter/status/1433123882366611466 https://twitter.com/benbalter/status/1433123882366611466
marderfarker2 5y ago
Was hoping to see some ideas on how to create a DMZ and it still being accessible locally. A use case would be to host internet facing applications at home and still be safely consumed locally.
- cmeacham98 5y ago
  I mean, the simple solution is to just access your DMZ via its public IP address. Alternative solution is to firewall traffic from the DMZ to the rest of the LAN but allow LAN to DMZ.
- carlhjerpe 5y ago
  For TCP this is simple, only allow new connections in one direction. The better way is to separate networks by vrf, ensuring packets take the right path and go through your dnat rules without possible shortcuts (firewall separation works too, but then you're at the mercy of not fat-fingering any rules)
brazzy 5y ago
I've been running a very similar basic setup (docker-compose + ansible + caddy + a bunch of other things, most importantly nextcloud) on a VPS for my personal server needs for about 2 years, so I'm feeling very validated right now.
lostlogin 5y ago
I’ve had good success against Google and Samsung (and maybe others) by setting a firewall rule whereby port 53 traffic can’t leave if it doesn’t come from my Pihole. I sent it back to the Pihole. There was a surprising amount of traffic that bypassed local DNS. It’s a strange feeling when you are actively fighting your own devices.
- judge2020 5y ago
  dhcp dns is just a suggestion.
- magicalhippo 5y ago
  I noticed an example of this when I set up split-horizon DNS for my local Home Assistant server. Apparently if Android gets an IPv6 address it expects an IPv6 DNS address as well or it will fall back to Google's own. Had to configure the PiHole with an ULA address and make the router serve that.
  - xxpor 5y ago
    Android's IPv6 support is the most actively user hostile thing I've ever seen. It's like if you took Apple's typical "our way or the highway" attitude and 10x'd it. For example, no DHCPv6 support: https://issuetracker.google.com/issues/36949085?pli=1 https://issuetracker.google.com/issues/36949085?pli=1 (note the date) They're so religious about it they wrote an entire RFC and made it a BCP so they could justify their silliness: https://datatracker.ietf.org/doc/html/rfc7934 https://datatracker.ietf.org/doc/html/rfc7934
- blowfish721 5y ago
  You might want to block port 443 to 8.8.8.8 & co as well to be on the safe side.
  - candiddevmike 5y ago
    Yes, DNS over HTTPS was designed specifically for this scenario. I wouldn't be surprised if trying to control DNS starts effectively bricking devices.
2OEH8eoCRo0 5y ago
This is similar to my setup that I host locally. I use an $80 laptop that I bought on craigslist and each service that I need runs in a rootless container under a single user just for that purpose. So far I have a dedicated Valheim server, Adguard, and Plex running under this scheme. I still have to move over my torrent client though. Perhaps I'll write some Ansible playbooks to make this easier to manage.
- gnu8 5y ago
  How do you know CIA agents didn’t sell you an implanted laptop on Craigslist?
  - 2OEH8eoCRo0 5y ago
    Because I am a CIA agent.
  - selfhoster11 5y ago
    How likely is that?
xoa 5y ago
I've gotten into networking and very much enjoy seeing how others approach the problem space at the scale level! It's gotten more important with not just the explosion of online threats, smart home devices etc., but also just because there is more and more value to be extracted from a high bandwidth WAN link via your own private services. That said I definitely strongly recommend against using anything UniFi based for routing/gateway tasks at this point (author mentions their AIO custom hardware UDM thing). They have significant issues now in WiFi and even switching, but at least remain kind of workable there. But the company has been an absolutely depressing development dumpster fire for years now and the trajectory still appears aimed straight down. And for whatever reason routing/gateway in particular is an area that only ever briefly got much oxygen there. So for a greenfield situation I'd be wary. There are lots of enormously better solutions for that. Personally I've dumped all my UniFi gateways (including some of the "next generation" ones I had for testing) and moved to OPNsense running on decent SuperMicro 1U edge systems (excellent value, quiet, regularly available dirt cheap on ebay and the like). But there are many great options there including VyOS, OpenWRT, or even just running straight OpenBSD if that's what you like. Router/gateway is one area where I thinking devoting real metal, preferably with higher reliability hardware and extra remote management options (like IPMI), is worth it. It's an important linchpin for a typical network, and while virtualizing it is possible and definitely desirable at large scale it's not where I want to add more moving parts without a team behind it. Everything else can go tits up while I mess with it or there are odd interactions and as long as the gateway/routing is still solid odds are I can recover without much disruption or having to physically be there.
- adrianpike 5y ago
  I'm about to do a big revamp of my home network and would love any specific gear recommendations to research - are these the supermicro setups you're talking about? https://www.supermicro.com/en/products/embedded/fanless-and-iot-gateway https://www.supermicro.com/en/products/embedded/fanless-and-...
  - xoa 5y ago
    I've used one of those in a satellite location and liked it ok, but what I actually ended up getting generally were full 1U replacements for the USG-4/UXG from SuperMicro's 5018D series. I kept an eye on ebay and found a bunch of new 5018D-FN8Ts listed for ~$650 a pop. They've got dedicated IPMI (obviously this needs to be secured), and unlike HP or Dell they include virtual iKVM for free and a full license is only $30. Pricing on the others can easily add +$200 for what should be included functionality even if they've got a spiffier coat of paint, which is significant on a low end server device. The 5018Ds are not fanless, but running they drop down to a load speed that I can't hear over anything else I've got (like a POE switch, all of which except the tiniest models have fans). HP's Proliants in contrast are jet engines all the time even when they're pulling <50W. I'm not running a total silent environment and preferred to make a rack closet instead well away from any living areas, but I don't want loud noise and SMs have been fine. Of course, one can just replace the fans with 40mm Notcuas or the like too if desired. They've got Xeon processors that have a bit more oomph than the Atoms or Celerons and also means ECC memory, since again I really want to be able to rely on gateways to a reasonable degree (this has already identified one bad memory module out of the dozen I had around). I install OPNsense on ZFS on a cheap small decent NVMe drive (PNY CS1030 250GB is around $35) which still means fast boot and no concerns about all the logging or the like I could desire. I have one single site that is doing much heavier 10G+ routing and usage that I also wanted to mess with more intensive SDN and security with. For that last year I ended up picking the much beefier and much more expensive EPYC Embedded based 5019D-FTN4 and putting a Mellanox card in it. It's also extremely quiet and has been really impressive, but that's stupid overkill right now. Also, EPYC Embedded is currently still based off of Gen1, there was no Zen3 update due to not having low TDP given the way the chiplets were upgraded vs the IO chip. I expect Zen4 next year will see an upgraded Embedded platform that will essentially be a 3 gen leap forward, so at this point not the best time anwyay. There is no perfect solution IMO. Though probably nothing that'd throw off the typical HNer, OPNsense does have its warts, rough edges and missing bits (no Webauthn so no security keys for login for example). It's based off of a FreeBSD variant (soon to be directly off FreeBSD) with all that comes with that for better or worse. Like, OPNsense does have a user space plugin option for WireGuard (along with ZeroTier and so on), but WG has not yet made it to the FreeBSD kernel which in some situations could be an issue (countered by raw CPU in my case). But it's powerful, well maintained, overall fairly user friendly, has pretty solid documentation and getting started guides, and a nice community with a good mix of developers and some companies behind it. The company Deciso for example does offer a paid business edition and paid support options if it's desired. It does have DNS Blacklist options ala PiHole, stats/telemetry/IDS/IPS via built-in and 3rd party offerings like Sensei, etc. There are plugins for Let's Encrypt, FreeRADIUS and other handy functionality. Someone who is very familiar with Linux might find VyOS more worth looking at but with my background I found OPNsense reasonably pleasant to get into. The decision tree here does also depend on how much network functionality you want to have into your gateway/routing system vs how much to stick on a separate server elsewhere (maybe virtualized or as a part of a NAS). Gateways can be very minimal or can handle damn near everything on the network. There are straight forward tradeoffs there in terms of failure modes and complexity.
gerbilly 5y ago
I've just finished a network overhaul as well. I used similar techniques, except I preferred not to use any external DNS server¹, so I hosted a recursive DNS server (bind9) on a small fanless server (apu2d4). The DNS server can also block ads, similar to how pihole does it. 1. Why is everyone trusting cloudflare to centralize all their DNS queries? Even Firefox is migrating all their browsers to use them.
- SamuelAdams 5y ago
  Personally I’m using cloud flare because there’s not a lot of options. The only other major one most tech enthusiasts know about is 8.8.8.8 and that’s owned by Google. It would be good if there were more players in this space. A nonprofit might help here too, like Mozilla. Maybe they can run a public DNS server.
  - tallanvor 5y ago
    quad9 is run by a foundation. Most of it's donations come from corporate sponsors, but it's definitely a better option.
  - justusthane 5y ago
    I think you might be kind of missing the point here—if you run your own recursive resolver, e.g. unbound, you’re not relying on any public DNS server.
rixrax 5y ago
I never thought I would use Docker for 'production' in home network, but I recently completed a transition from three separate x64 servers to a single Raspberry Pi 4 [mostly] provisioned using Docker. Energy use-wise I went from maybe 90W to 2.9W. And to a single old-skool shell script that deploys and sets-up everything on a freshly installed Raspbian OS. In addition to the services mentioned in the article, one that I recently added to my home network is Stratum-1 NTP server that gets it's time via Pi GPIO GPS module. Whether or not this has any real positive privacy or security implications, I do like the idea that I have significantly reduced outbound traffic to/from port 123 from the devices in the home network. Interestingly most devices are happy with NTP server provided via DHCP, but notably Apple and Sonos products always want to go to time.apple.com and Sonos' alternative (and I haven't yet have had heart to setup split DNS to try and redirect them to local NTP anyways...). edit: typos
- OrvalWintermute 5y ago
  > I recently added to the home network is Stratum-1 NTP server that gets it's time via Pi GPIO GPS module. Whether this has any real positive privacy or security implications, I do like the idea that I have significantly reduced outbound traffic to/from port 123 from the devices in home network I like the way you think, and was thinking about some of the same. Could you share what solution you went with in terms of hardware? I am interested in using better than commodity time that could serve both NTP and precision time as well. The benefits for better time, and local time are fairly impacting if you are doing anything with time sensitive computing (PKI, performance measurement, and audio are interests of mine).
  - rixrax 5y ago
    I used this board from Uputronics[0] with optionally included active GPS antenna. Mostly because they were readily available, and there were few guides online for setting ntpd up on Pi using Uputronics board. As a word of caution - it looks like most guides that I came across are outdated, or indeed have never produced a reliably (or at all) working NTPd setup. Instructions that lead me to the 'right' path is linked here[1]. I am still in the process of tuning GPS offset in my setup. But all in all, it's been a great learning experience into complexities of the protocol I've mostly taken for granted for a very long time. [0] https://store.uputronics.com/index.php?route=product/product&product_id=81 https://store.uputronics.com/index.php?route=product/product... [1] https://www.philrandal.co.uk/blog/archives/2019/04/entry_213.html https://www.philrandal.co.uk/blog/archives/2019/04/entry_213...
OrvalWintermute 5y ago
This doesn't strike me as particularly secure, private, or overengineered considering his position and level of access.
- goodpoint 5y ago
  overengineered for sure. Secure and private - no.
djanogo 5y ago
Running something similar with UDM on HP Prodesk G2 which I bought for $50. Ubuntu server with HomeAssistant VM and Portainer. Uses about 7w, CPU temp 32C. I was running Pi4 before but felt it was WAY overrated for this purpose, you can buy a used micro computers for $150 which come with SSD, 8GB RAM and are multiple times faster than Pi4.
BrandoElFollito 5y ago
If someone is interested, some comments on my home setup (about 4-5 years of continuous tinkering) I have FTTH (France) and the first thing I did is to replace the ISP-provided router. They are really poor quality. I bought a Ubiquity ER-4 and, retroactively, I should have bought a mini fanless PC and run Debian on it. Why Deian and not OpenSense or something like that? Because I have control over the very limited services I will use on the router, namely routing, firewalling and dnsmasq for DHCP and DNS. It is difficult to keep up with the security / functionality otherwise. Teh worst is a closed archaic system such as the one on the Ubiquity ER-4. A step up is something more dynamic such as OpenSense or similar. If you have the will to learn a bit of Linux Debian is the cleanest solution. Again - this is really for a very small set of service you will almost never touch. Behind the ER-4 there is a switch that powers a few end user PCs, a PoE port for a Ubiquity Unifi WiFi access point and a server The Unifi AP is fantastic. This is such an extreme upgrade vs the ISP provided one that I cannot imaging going back. Really invest in a higher-range consumer AP. In my apartment I used to have along the rooms (more or less linearly) 4, then 2 then 1 and 0-1 "wifi bars" (roughly the quality of reception). I now have, for the same emission power, 4 4 4 3 Then the server. There are two basic parts on it: the "services" part and the "home automation" part. The services are things (web sites, MQTT, ...) that are run exclusively from Docker. I tested the recovery of the whole system from only the backups (of the configuration (docker-compose.yaml) and the non-volatile volumes) and it took me 1 hour from "I do not have any documentation, nor ISO" to "the lights work again"). Docker is a life saver here. Backups with Borg. The second part on the server is home automation: a docker container for Home Assistant and a USB dongle for Zigbee. I then have Zigbee and Wi-Fi devices all over the appartment to get rid of fixed wall switches. Plus automation. What I would have done differently? A mini PC instead of the ER-4, two electrical wires in the wall (phase and neutral). What I will do in the future? A redundant setup for home automation (this is not simple at all). I would love to get some feedback for those who are more experienced or went though things they would do differently, or again.
- willis936 5y ago
  I have a pretty similar hardware setup but I went for a used Ruckus R610 WAP. With a PoE injector and shipping I spent around 200 USD. I'm really happy with it. My current goals are proper IPv6 support in case the day comes where my ISP will no longer give me an IPv4 address. The setup is currently close to perfect, but my DDNS service (google domains) isn't letting me make dynamic AAAA records for some reason. I'm in talks with support right now. I also have not successfully made ip6tables entries for rerouting IPv6 DNS traffic to my local DNS servers. When I have made entries it totally breaks IPv6. I too would prefer an x86 box for a router, but OTOH purpose-built routers are much more energy efficient. Heat is bad for reliability and more power means less run time on a UPS. My next home network purchases will be an all-flash TrueNAS box that is backed up to Backblaze. I want to get off privacy-devoid cloud services.
Johnny555 5y ago
Does anyone make an AP that allows unlimited (or at least a lot) of SSID's tied to separate VLANs? My Unifi's only support 4, so I have my private network (for my own laptops, etc), guest network, camera network and IoT network. But I'd love to be able to separate IoT devices across multiple networks, so I could have one for smart switches, one for TV/media players, etc.
- rand49an 5y ago
  All enterprise gear should allow you to have multiple SSIDs on different or dynamic VLANs. You won't have that issue on Meraki, Ruckus, Cisco or Aruba gear. It's worth noting though that more than 3 SSIDs brings with it radio overhead which can degrade performance of your wireless network.
  - Johnny555 5y ago
    I'm fine with losing Wifi performance if I can improve my home network security. When I'm sitting at my desk, my laptop is hardwired, and my only 4K TV is hardwired, so I don't have any great bandwidth needs, and even with 4 SSID's broadcast, I'm maxing out my 500mbit internet connection, so I even if I lost half that to overhead, I'd be ok with that. I've also thought about just setting up multiple nodes, one set for secure devices, one for insecure devices. Cisco "only" supports 16 SSID's, which is probably more than I'd even need.
- EvanAnderson 5y ago
  You don't want "unlimited" SSIDs-- you want a RADIUS-assigned VLAN and a single SSID. The credentials used to associate determine which VLAN the traffic dumps into. Your UniFi gear will do it. Of course, a lot of consumer-grade hardware won't do 802.1x so you end up stuck with needing a bunch of SSIDs (and wasting air-time on beacons).
  - Johnny555 5y ago
    Of course, a lot of consumer-grade hardware won't do 802.1x so you end up stuck with needing a bunch of SSIDs (and wasting air-time on beacons) Yes, that's why I want more SSID's. My internal network does use 802.1x, but as you said, few devices outside of laptops support it.
LgWoodenBadger 5y ago
From a physical security perspective, I cannot recommend and emphasize enough having a fiber optic media converter between your network interface (cable modem, satellite, etc.) and your expensive and extensive network. The purpose is to isolate your expensive hardware from lightning-induced destruction. The cable modem and one of the media converters serve as "sacrificial anodes" in this setup. It ends up being much cheaper to replace just a cable modem and a single media converter instead of an entire network of devices.
- walterbell 5y ago
  After losing a cable modem and the connected NIC port to lightning in urban apartment, I installed a lightning suppressor / surge arrester with a replaceable gas discharge tube, on both the coax cable to the cable modem, and the ethernet cable to the modem. Since then, I've once replaced the coax-facing tube/fuse due to a surge, but the equipment has been fine. Spare fuses were back-ordered at Mouser, but arrived eventually.
- mongol 5y ago
  That is a great point, but what about power? What do you recommend there?
  - vorpalhex 5y ago
    Surge protector (high quality, insured) or UPS.
egberts1 5y ago
wow. Just dropped in a new home gateway (Devuan 3/Debian 11) - hardened Linux (Kicksecure/Whonix) - openrc (no systemd network vulnerabilities) - DMZ - NAT + IPv6 - stealth/hidden master DNS server. - Shorewall firewall - Wireless, 4 SSIDs - Wireguard to remote VPS carrying DNS queries DMZ covers all IoT, Alexa, SmartTV, game console, cable TV; both by wireless and CAT5e. Household network covers laptop, desktop, cloud servers (Proxmox), Kerberos ticket server, file server (NFSv4, media DVR, ownCloud server. And my white lab is totally airgapped and occasionally waterfalled firewall as needed. I have a subnet that offers encrypted MAC.
- Jayakumark 5y ago
  Do you have the instructions somewhere to follow
  - egberts1 5y ago
    I’m writing them up as I go along. Need to break them up into short blogs. Now doing transparent interception squid. And I bookmarked this comment so I can reply to you later
    - Jayakumark 5y ago
      Appreciated, thanks
lambic 5y ago
I recently installed adguard home on my network because some googling suggested that was better than pi-hole. Is there a reason you picked pi-hole?
- benbalter 5y ago
  PiHole is what first introduced me to the DNS sinkholing concept and was more mature when I was first researching options. AdGuard home has come a long way since then and I’m planning on giving it a closer look when I’m looking for my next project.
SavantIdiot 5y ago
His research boils down to two things: 1. Lock down DNS at all costs (and he means, all costs), going so far as to use PiHole and then ridirecting DoH/DNS to CloudFlare to scan for more mal sites. 2. Create virtual subnets at home for untrusted IoT devices
- HighPlainsDrftr 5y ago
  This. I'm really concerned that these IoT devices will begin hard coding their own setup and using DoH to get around any DNS restrictions. I've started hardwiring devices that I have some trust in (eg laptops, desktops, etc) on to their own VLAN. I really want to setup a second wifi network, maybe a third for Rokus, IoT and other things, and force them through a proxy. Or they just don't get access to the Internet at all. It seems silly that these companies think that can use my bandwidth, whether or not it's metered, to do as they wish.
chaganated 5y ago
Where can I buy put options on github's network security?
g8oz 5y ago
Any recommendations for a dedicated OpenWRT box for the home? Links welcome. A few 2.5 GB ports would be nice.
c7DJTLrn 5y ago
It's bizarre to me - the idea of running so many things on a home network that you need to start writing Ansible playbooks, building Docker images, spinning up Kubernetes clusters, putting a rack together. I think if you really applied some thought you could get rid of all of it and save yourself the time and effort. But I guess tech people do have a tendency to find nails for their hammer.
- darkwater 5y ago
  It's just a hobby, it gives you some extra nice thing you wouldn't have that easily (for one, adblocking at DNS level with PiHole) but in the end is just a hobby.
  - c7DJTLrn 5y ago
    I have nothing against hobbies but it's even better if your hobby has something to show for it. I think the time saved by just manually plonking the software on a cloud VPS would allow the author to do something more challenging and interesting, instead of well... pleasuring oneself with heavy duty deployment tools.
    - kipari 5y ago
      Heavy duty deployment tooling is a hobby of its own. :-)
    - laumars 5y ago
      Reading a book doesn’t give you anything tactile to show for it. You read for the experience. Likewise for gaming, solving a crossword, nor watching TV. Different people relax in different ways.
    - Godel_unicode 5y ago
      This is pretty condescending, and also misses the major point made in TFA. Writing a quick Ansible playbook/docker compose means you can quickly standup your services on any system. Run it locally on a raspberry pi, move it to a VPS, just run one command. None of this tooling is "heavy duty" by any means. If you just apply a little thought, you'll see how this saves time and effort.
lostmsu 5y ago
I am wondering if I'd prefer to use NixOS to using Docker + Ansible + yada-yada for a similar setup. I've never used either, just a guess after reading many articles.
unixhero 5y ago
A drawing would help!
ncmncm 5y ago
Ansible looks useful, but YML is a hellscape file format. Is there a system equivalent to Ansible that uses a decent config file format?
- Notanothertoo 5y ago
  Terraform?
- beefjerkins 5y ago
  What would you consider to be a decent config file format?
  - ncmncm 5y ago
    TOML seems OK.
Havoc 5y ago
Fairy similar to what I did except I used proxmox and lxc not docker compose. Since proxmox is addressable via cli you can ansible against it to deploy stuff. There is also a module for it but I skipped that. I’d suggest blocking port 53 at the firewall too. I was surprised how much stuff doesn’t go through the cloudflared tunnel. You think it’s all going through the Pihole but there is so much rogue stuff
xg15 5y ago
On the whole Caddy/TLS bit: Did I miss something or wouldn't he also have to purchase a domain for the admin interface? (And then setup pihole such that the domain is resolved to the pihole server itself inside the LAN). Of course it might be that he already had a personal domain available from some other project, that he could use. But actually buying a domain just so your raspberry pi can use https seems like the ultimate overengeneering.
- thedanbob 5y ago
  You're correct, but you can see he already has a personal domain hosting his blog: ben.balter.com. He probably just pointed another subdomain at the pihole e.g. pihole.balter.com. That's what I did with my piholes and other internal web apps on my home network.
aaroninsf 5y ago
De rigeur despairing comment, which will be downvoted into obscurity: That such efforts are necessary, and as the commentary shows, an active area of interest and hacking and development by so many very smart people, is an absolute condemnation on the dystopian state of the surveillance capitalism that so many of us here have helped build. It's not malicious actors that are the sole or even primary vector being blocked. It's the now-systemic misbehavior of so many of our own products and services.
- selfhoster11 5y ago
  I'm not sure what you're even trying to say here.
  - aaroninsf 5y ago
    I'm saying that it's well worth taking a step back from the tree-level view, e.g. celebrating and being inspired by the intellectual and technical details of a smart industrious individual attempt to reasonable degree insulate themselves from the abuses of surveillance capital, and consider the forest-level view, that while few were paying much attention, our society in general, and the industry most HN readers contribute to in specific–in some cases, very very directly–is now profoundly, all but irredeemably broken, wrong, amoral, and bad, pick a word. Articles of this type and fandom and advocacy of this or that open source solution to a tiny piece of the surveillance capitalism nightmare, are in small way part of the problem; they normalize the experience well sketched here: https://den.dev/blog/user-hostile-software/ https://den.dev/blog/user-hostile-software/ But that article only approaches some sides of the problem. The political and philosophical ones, which are implicit but not blown out, are deep, real, and corrosive. The subtext, which is not so subterranean, is that readers here who are cashing a pay check accelerating total information awareness, dark patterns, and the abuse of users in exchange for calculatedly seductive or addictive services whose fine print all know no one ever reads, are culpable, and that damage is real, and the costs real. And they should do better.
noncoml 5y ago
"How I re–over-engineered my home network for privacy and security" Proceeds to send all his DNS queries to Cloudflare
- KeepFlying 5y ago
  Still better than sending them to his ISP (or keeping them visible to his ISP no matter the destination). Sure it's not ideal but really very few things are when it comes to home network DNS requests. I'd love suggestions for better solutions though, I'm sure there's something I haven't considered.
  - noncoml 5y ago
    I am a hypocrite as I do the exact same thing, Pi-Hole + Cloudflare. It's just that your title raised my hopes for a better solution. Sorry :)
  - Skunkleton 5y ago
    I intercept port 53 traffic at my router, tunnel it over to my VPS which runs an unbound instance. Overall I think this setup is better because: 1. It denies my ISP the chance to look at my DNS requests. 2. It doesn't involve yet another third party which may or may not try to monetize my dns requests. 3. It mixes my dns related traffic with that of a datacenter, which doesn't provide a compelling source of data for advertisers IMO. DoH throws a wrench in the works. Still considering my options on how I want to address it.
firecall 5y ago
Nice article, but not really over engineered. More, spent money on a bunch of gear and installed PiHole! I did the same though LOL However, in the spirit of his comments about Outsourcing to The Experts, I switched from PiHole to AdGuard Home to NextDNS. Very happy with NDNS, just wish there was a kill switch! I'm up around 50 devices on my home network, so the standard home router from the ISP, or even the fancy 'gaming routers' that look like alien spiders don't really cut it! I get terrible internet where I am in Australia, so I have to be creative: ---------- My Setup: ---------- • 3x Unifi UAPs • 1x USG Pro • 1x USW 16 POE • 2x 4G modems Connected to the USG Pro and load balanced! You can't LB with a UDM! •1x M1 Mac mini with 12TB RAID as a TMS and Content Cache! Serves around ~500GB a month to my network!
devnulll 5y ago
I've done a similar setup at home with a full Unifi setup, PiHole, DNS over TLS, Let's Encrypt certs for internal devices, multiple wifi networks and other "It's COVID and I'm bored" goodness. I've had to turn most of it off. PiHole means every single web site issue for any user on the WiFi ends up on my plate immediately. Wifi network segments mean Minecraft doesn't "Just work" for the kids and their friends. IOT networks means Airplay doesn't find devices. SDDP doesn't work, etc. Having a functional home network turns out to be more important than having a secure home network.
bacan 5y ago
This post makes no sense, at all. 1) UniFi devices just recently had a massive security vulnerability. Also not a good idea to be letting third party servers, access your home network, directly via your router 2) Instead of CloudFlare, use something like Unbound. I have mine setup to fetch directly from the Root servers. Why send your DNS queries upstream? 3) Caddy is the one decent thing suggested, however I am skeptical of the benefits of having HTTPS internally in your LAN. If an attacker is in your LAN, then it is already Game Over
dvcrn 5y ago
I went down a similar hole with a UniFi Dream Machine. udm-utilities [0] basically allows you to run all kinds of stuff on it through podman. Went from trying to teach it to mimic my ISPs router to get internet working, to banging my head against IPv6 prefix delegation, to now running adguard, homebridge and a bunch of other things across different VLANs on it. All in all, I am happy with the result, and happier that I got dragged out of my progrmaming bubble to learn proper networking with IPv6. [0]: https://github.com/boostchicken/udm-utilities https://github.com/boostchicken/udm-utilities
8fingerlouie 5y ago
Sounds like my old networking setup, except i had a special "IoT like" VLAN for my kids and their friends. No reason for malware to propagate outside the kids network :) Besides that was a DMZ and a storage VLAN, and because i'm cheap i opted for a router on a stick kinda setup with all storage being mounted across the firewall as kerberized NFSv4. Ideally i would probably have setup an L3 switch with ACLs, but i had the processing power / bandwidth on the router. These days i've taken it at step further (back). PiHole got replaced by AdGuard Home, which then in turn got replaced by NextDNS. PiHole/AdGuard Home only works on your own network, and NextDNS works everywhere. The price they're asking ($20/year) is less than the cost of the Raspberry Pi and power required to run it, and requires no maintenance on my end. Everything that previously ran in docker at home has been replaced with Cloud equivalent services, source encrypted by either rclone + encryption, or just using Cryptomator. Again, the cost of the cloud services is cheaper (about half) of what i paid in power consumption for just running the server. Then add hardware costs on top of that. As for my home network, since everything is now "out there", i've retired about half my network infrastructure (yay, less stuff that can break), and the internet connection is now the lower common denominator. I used some of the money i saved on power consumption to instead upgrade my internet connection to a 500/500. All that remains is a single Mac Mini that is powered 24/7 for the purpose of pulling our iCloud data back home so that i can make backups. I would really love if TimeMachine or 3rd party backup tools could backup that cloud data directly from the client, downloading it only once, and other than that leave it in the cloud.
SavantIdiot 5y ago
UniFi now offers L2 isolation on WLANs, so I don't think he needs a VLAN for the IoT wifi. How do you create a rule to isolate a printer? It still needs to receive requests from the network, but I don't want it probing other devices...?

345 points by chmaynard 5y ago

For anyone who wants to go a similar route I can highly recommend OpenWrt as the OS. Not only does it run on cheap routers, but also on big x86 machines and even in VMs and/or containers. For example, I run OpenWrt on my desktop in a LXC container to manage networking (Wifi, bridging firewall for bridging VMs into the network, general firewall, etc.) through the nice webui. It gets direct to access the WiFi adapter and the host gets access through a bridge connected to the container via a VETH interface. It comes with nice addon packages for stuff like Wireguard, all kinds of tunnels/VPN, adblockers, runs containers and a ton more. I even run it on a VPS as container with it having exclusive access to the "physical" NIC. The parent OS isn't directly accessible at all. Makes firewalling a breeze. The only open ports are for the Tor relay and Wireguard, through Wich I connect to the webui/ssh and do everything else. Of course, my router also runs OpenWrt...

atatatat 5y ago
> big x86 machines You saw the "security" in the title, right?
carlhjerpe 5y ago
How is the OpenWrt CLI? Say I want to deploy it with Ansible. How does it stack up against VyOS (which just recently got VRF-lite support)
- dtx1 5y ago
  > How is the OpenWrt CLI You SSH into it. Then do whatever you need to. There's nothing in the UI that can't be done via CLI as far as I know. Some plugins might not be 100% CLI compliant but at least the Base UI (luci) is completely transparent to the CLI via uci.
  - HighPlainsDrftr 5y ago
    Do you have any recommended hardware for OpenWRT? I've been wanting to put in a low powered router/firewall on my home network that isn't controlled by a big vendor. I haven't done a ton of research in this area, but I'd certainly like to use OpenWRT or OPNSense on my home routers/firewalls. Side note: I've been trying to figure out a decent way to get rid of Android on my Galaxy S9+, but it appears to be locked. At the end of the day, I just want to be in control of my bandwidth, my data, and know whats going on. Big companies are making this very complicated with all the tracking. I recently re-enabled my pi-hole on a virtual machine, and it never ceases to amaze me what is talking to the internet without my permission. After digging into DoH a bit, I'm about to the point where I think I need to put in an outbound proxy, deny all outbound access except via the proxy, and iterate again and again. I just don't want to have $200 a month in power bills to support my home network to save bandwidth and know what is traversing the net.
    - dtx1 5y ago
      WRT3200ACM is what I use. It's one of the most powerful consumer devices with wifi supported by openwrt. It's powerful enough for a router with a little firewalling, VPN (wireguard only if you want speed), DoH and some Ad Filtering but thats pushing it's limits from my experience. If you want more power https://openbsdrouterguide.net/ https://openbsdrouterguide.net/ is your friend. That being said there is no hardcore prosumer hardware out there for this purpose. The moment you go beyond home user router hardware like the WRT3200acm you are in either CISCO Buisness stuff or custom server builds. Potentially a Raspberry Pi 4 with a PCIE ethernet card is closest to prosumer hardware out there and there's a lot of hacking involved to get that running to the same degree as a openwrt router
      - HighPlainsDrftr 5y ago
        This is interesting. I'm going to research it further. I really appreciate the feedback. I'm really starting to hate DoH - and I may just not put any IoT thing on my network that uses it. Maybe that's the way to go. But I doubt most consumers really care. It's complicated.
- rhn_mk1 5y ago
  OpenWRT doesn't pack Python out of the box, so you need to install it, and have enough space for it, if you want to use Ansible. It uses a custom stack for configuration (like VyOS), so builtin ansible tasks won't always be so helpful. Configuration is not stored in a single place, but in several files, and it's easy to lock yourself out while testing changes: there's no commit-timeout, and there's no committing of changes, nor rollback. It's just editing random files, and restarting services. I think there's a special configuration command that might fix some of the above issues, but I've been using the web interface (which actually does support committing and, to some extent, validation).
OrvalWintermute 5y ago
I thought about which OS for some of the same things and I realized that I would rather go with a lab version of a full enterprise firewall. A Palo Alto VM gets you pretty much most of the sweet PA features without the cost, and a better approach than an outdated strategy like VLAN as Access Control, or zone firewalling, permitting the use of permit/deny by protocol, and overall better privilege tiering by network area.
- EvanAnderson 5y ago
  > A Palo Alto VM ... without the cost ... Does Palo Alto have some kind of no-cost offering in their VM line?
  - OrvalWintermute 5y ago
    > A Palo Alto VM ... without the cost ... This isn't free, but $50-200 is alot less than $2-4k
    - EvanAnderson 5y ago
      Yikes! I didn't realize PA had anything available for less than a 4 digit price point. I'll check it out. Thanks!
- todd8 5y ago
  I’m curious about PA firewalls. The product descriptions claim “Machine Learning” based routing/firewalls. What does that even mean? I’m a bit skeptical about AI being used in a firewall. Can someone help me understand why I should consider this instead of running pfsense on a Netgate appliance?
eric__cartman 5y ago
I always found it baffling how a community maintained project like OpenWrt can not only keep routers updated for many years (unlike the manufacturer that gives you an update once in a blue moon for the first 2 years) but make these cheap routers so useful and so stable you just forget the thing is there in the first place. I can't remember the last time I had to reboot a router running OpenWrt because it started behaving erratically. You can do a lot with these commodity ARM CPUs, 64-128MB of ram and a few tens of megabytes of flash storage.
- maerF0x0 5y ago
  i literally have my comcast router on a 24 hour time and it shutsdown for an hour each night from 3-4am . Communism style brownouts of internet over night
123pie123 5y ago
for home I can recommend openwrt running on a BT Home Hub 5A you can buy them in the uk/ ebay for 15-20 pounds already with openwrt installed (you can do it yourself, but it includes a bit of soldering) - I have two in case the main one fails - talks to most if not all ISPs I love openwrt now, it does take a bit of getting used to if you havent used it before. I mainly use to lock my wifi down between hours for the kids. whilst keeping another wifi/ SSID open. for security all my NAS's are wired and locked down to key wired computers - I keep meaning to create a Nextcloud gateway on docker
stjohnswarts 5y ago
I have dd-wrt on a tp-link router and it works great and has been for about 3 years now. No issues. I just have to check the site about once a month and see if there's an update. I wish that was automated or had a notice in the web gui a new one is available. Very configurable and stable.
3np 5y ago
You know, I used to do this and give the same advice as well but after a lot of time spent on it, I am not anymore. The way OpenWRT handles routing and firewall rules is particular and they apply their own terminology for some things. They have their own distro-specific packages for things like DHCP (odhcp(c)d) and firewall (fw3). For very simple networks, it's very smooth to get to where you want. Add on dual-stack v4/v6, vlans, multiple firewall zones, routing policies etc and things start becoming very unpredictable. Oh, and that adblock package? Turns out a single invalid line in a blocklist will completely break DNS (at least on the version I was running from last year). Not to mention that (AFAIK) there's no good way to keep up to date with security patches and bugfixes while keeping the system stable. After all the countless hours I poured into OpenWRT configuration, I finally realized that it's so much less pain and confusion with vanilla Debian with systemd-networkd (which BTW natively supports setting up Wireguard interfaces now) and firewalld+nftables, everything configured via ansible playbooks. For someone diving into this today, it's a lot easier and more future-proof with nftables than iptables - and OpenWRT will be married to iptables for the foreseeable future. It's great that it works for you, but if you like I did have some imposter syndrome over not perfectly understanding Linux networking and are happy that OpenWRT takes care of those confusing iptables rules and routing policies and what-not - you may just discover that learning how it actually works will take less work than abusing OpenWRT into doing what you want. Sure, you have to give up the WebUI and some of the custom add-ons. I am sure BSD or Rocky Linux are fine choices as well; Debian just happens to be what I mostly use for servers otherwise. I don't want to hate too much on OpenWRT as it's great for novices with trivial needs and there are many devices where it or dd-wrt are the only readily available options. But if you run Linux anyway and have an x86/amd64/arm device you're going to use as a main router, I'd recommend choosing a "normal" distro and setting things up from scratch.

syntaxing 5y ago

Some of the authors concerns with Pihole can probably be fixed by replacing it with Adguard home because of the HTTPS support. That being said, it’s kind of cool to see someone higher on the corporate ladder posting these kind of blog posts!

nickphx 5y ago

I would hardly consider blindly trusting a third party like cloudflare to be over-engineering.

traceroute66 5y ago
"I over-engineered my network for privacy" .... by trusting a third-party for DNS resolution. Doesn't compute ! Even more so as the OP chose Cloudflare ! US Jursisdiction + Commercial Company ?
- lostlogin 5y ago
  Have you got a recommendation? I see them as a good choice compared to my other options. but if there is better out there…
  - ignoramous 5y ago
    Use DNSCrypt with anonymized relays: https://github.com/DNSCrypt/dnscrypt-proxy/wiki/Anonymized-DNS https://github.com/DNSCrypt/dnscrypt-proxy/wiki/Anonymized-D... Or, wait until ODoH goes mainstream: https://datatracker.ietf.org/doc/html/draft-pauly-dprive-oblivious-doh-06 https://datatracker.ietf.org/doc/html/draft-pauly-dprive-obl...
  - traceroute66 5y ago
    > Have you got a recommendation? Well, if you're going to go to the trouble of building and running your own VMs or docker images then the obvious recommendation is to run your own recursive resolver (Unbound or Knot Resolver). The added bonus is that CDN content will operate as expected for your IP range instead of you talking to the CDN server nearest to whatever datacentre you spoke to the third-party DNS BGP anycast on. This may or may not be a big deal depending on your geographic location and internet connection. I am aware some third-party DNS services have options to try to fix the CDN problem, but YMMV as to whether this works in practice. Otherwise, if I was forced to name a third-party, I would likely name Quad9. Especially after their recent relocation to Swiss jurisdiction[1]. IIRC (its been a while since I read it) their privacy policy is also better than Cloudflare's too (e.g. no weird, "we'll retain some stuff for 25 hours" IIRC). [1] https://www.quad9.net/news/blog/quad9-public-domain-name-service-moves-to-switzerland-for-maximum-internet-privacy-protection/ https://www.quad9.net/news/blog/quad9-public-domain-name-ser...

late_groomer 5y ago

Really glad to see this write-up. I love my current PiHole setup, but replicating it for family/friends (especially with my blocking so many sites they might want) hasn't seemed doable. I would be having to help them constantly with blocked sites, etc.,. My setup could be even better if either Fios or Orbi provided a half-decent router, but just using PiHole as my DNS server has been awesome. When my Pi (SD card) crashed and I had to revert to my old router setup, it was shocking how slow everything was. I'd become so used to pages loading in a blink, waiting a second or two while ads loaded seemed to take forever. If you have any unused SFF computer gathering dust, give PiHole a try (runs on any of the major linux distros). Initial setup takes under an hour. You can customize the backend out of it with blocksite lists; I'm blocking so much my wife and kids can't even install new iPhone apps. Amazon Devices, Rokus, Google, nothing gets to call home unless I allow it. I have it forcing all Google/Bing searches to use their clean filter; worrying my kids might google "porn" is a thing of the past. There are easy to find tutorials out there, just be sure to filter Google images, as well.

dreyfan 5y ago
That sounds overbearing and your mobile data bills will skyrocket as nobody wants to use your annoying home setup.
- lostlogin 5y ago
  I’ve had more than one person ask how to replicate mine after the standard internet noise was removed while on my wifi. It’s also fairly easy to keep a VPN to home running to avoid nosy cooperate wifi and enjoy my own vicious filters.
dawson 5y ago
Did you consider using NextDNS?
creeble 5y ago
FWIW, a plug here for The FreshTomato router firmware. I just retired my PiHole because FT does the same job seemingly just as well. Tomato has long been my chosen router firmware - much easier to use than OpenWrt, but not quite as feature-laden and runs on a more select set of hardware (mostly ASUS routers). But it has recently gotten a development boost from a new maintainer, and the ad-blocking seems just as good as PiHole. I suppose I can unplug my PiHole now that there’s no traffic going to it...

ctur 5y ago

This is a great write-up, not just for the ostensible goal (more privacy and security with maintainable infrastructure) but also because it lays out some reusable primitive for other “home lab” type projects. The next stop down this fun slippery slope is kubernetes :) That said, it also is worth mentioning NextDNS as an option. It basically is “pihole as a service” and is reasonably affordable. I ended up going that route to achieve similar goals as to the author and have been happy with it.

ignoramous 5y ago
> That said, it also is worth mentioning NextDNS as an option. There's ControlD.com as well (by Windscribe) that's super neat which I really like, that offers more than just DNS. Disclosure: I co-develop a FOSS NextDNS alternative.
wintermutestwin 5y ago
Doesn't NextDNS have the same problem as VPN where you are trusting all your browsing data to one entity?
aheckler 5y ago
I asked the author of the OP about NextDNS on Twitter. His reply: https://twitter.com/benbalter/status/1433123882366611466 https://twitter.com/benbalter/status/1433123882366611466

marderfarker2 5y ago

Was hoping to see some ideas on how to create a DMZ and it still being accessible locally. A use case would be to host internet facing applications at home and still be safely consumed locally.

cmeacham98 5y ago
I mean, the simple solution is to just access your DMZ via its public IP address. Alternative solution is to firewall traffic from the DMZ to the rest of the LAN but allow LAN to DMZ.
carlhjerpe 5y ago
For TCP this is simple, only allow new connections in one direction. The better way is to separate networks by vrf, ensuring packets take the right path and go through your dnat rules without possible shortcuts (firewall separation works too, but then you're at the mercy of not fat-fingering any rules)

brazzy 5y ago

I've been running a very similar basic setup (docker-compose + ansible + caddy + a bunch of other things, most importantly nextcloud) on a VPS for my personal server needs for about 2 years, so I'm feeling very validated right now.

lostlogin 5y ago

I’ve had good success against Google and Samsung (and maybe others) by setting a firewall rule whereby port 53 traffic can’t leave if it doesn’t come from my Pihole. I sent it back to the Pihole. There was a surprising amount of traffic that bypassed local DNS. It’s a strange feeling when you are actively fighting your own devices.

judge2020 5y ago
dhcp dns is just a suggestion.
magicalhippo 5y ago
I noticed an example of this when I set up split-horizon DNS for my local Home Assistant server. Apparently if Android gets an IPv6 address it expects an IPv6 DNS address as well or it will fall back to Google's own. Had to configure the PiHole with an ULA address and make the router serve that.
- xxpor 5y ago
  Android's IPv6 support is the most actively user hostile thing I've ever seen. It's like if you took Apple's typical "our way or the highway" attitude and 10x'd it. For example, no DHCPv6 support: https://issuetracker.google.com/issues/36949085?pli=1 https://issuetracker.google.com/issues/36949085?pli=1 (note the date) They're so religious about it they wrote an entire RFC and made it a BCP so they could justify their silliness: https://datatracker.ietf.org/doc/html/rfc7934 https://datatracker.ietf.org/doc/html/rfc7934
blowfish721 5y ago
You might want to block port 443 to 8.8.8.8 & co as well to be on the safe side.
- candiddevmike 5y ago
  Yes, DNS over HTTPS was designed specifically for this scenario. I wouldn't be surprised if trying to control DNS starts effectively bricking devices.

2OEH8eoCRo0 5y ago

This is similar to my setup that I host locally. I use an $80 laptop that I bought on craigslist and each service that I need runs in a rootless container under a single user just for that purpose. So far I have a dedicated Valheim server, Adguard, and Plex running under this scheme. I still have to move over my torrent client though. Perhaps I'll write some Ansible playbooks to make this easier to manage.

gnu8 5y ago
How do you know CIA agents didn’t sell you an implanted laptop on Craigslist?
- 2OEH8eoCRo0 5y ago
  Because I am a CIA agent.
- selfhoster11 5y ago
  How likely is that?

xoa 5y ago

I've gotten into networking and very much enjoy seeing how others approach the problem space at the scale level! It's gotten more important with not just the explosion of online threats, smart home devices etc., but also just because there is more and more value to be extracted from a high bandwidth WAN link via your own private services. That said I definitely strongly recommend against using anything UniFi based for routing/gateway tasks at this point (author mentions their AIO custom hardware UDM thing). They have significant issues now in WiFi and even switching, but at least remain kind of workable there. But the company has been an absolutely depressing development dumpster fire for years now and the trajectory still appears aimed straight down. And for whatever reason routing/gateway in particular is an area that only ever briefly got much oxygen there. So for a greenfield situation I'd be wary. There are lots of enormously better solutions for that. Personally I've dumped all my UniFi gateways (including some of the "next generation" ones I had for testing) and moved to OPNsense running on decent SuperMicro 1U edge systems (excellent value, quiet, regularly available dirt cheap on ebay and the like). But there are many great options there including VyOS, OpenWRT, or even just running straight OpenBSD if that's what you like. Router/gateway is one area where I thinking devoting real metal, preferably with higher reliability hardware and extra remote management options (like IPMI), is worth it. It's an important linchpin for a typical network, and while virtualizing it is possible and definitely desirable at large scale it's not where I want to add more moving parts without a team behind it. Everything else can go tits up while I mess with it or there are odd interactions and as long as the gateway/routing is still solid odds are I can recover without much disruption or having to physically be there.

adrianpike 5y ago
I'm about to do a big revamp of my home network and would love any specific gear recommendations to research - are these the supermicro setups you're talking about? https://www.supermicro.com/en/products/embedded/fanless-and-iot-gateway https://www.supermicro.com/en/products/embedded/fanless-and-...
- xoa 5y ago
  I've used one of those in a satellite location and liked it ok, but what I actually ended up getting generally were full 1U replacements for the USG-4/UXG from SuperMicro's 5018D series. I kept an eye on ebay and found a bunch of new 5018D-FN8Ts listed for ~$650 a pop. They've got dedicated IPMI (obviously this needs to be secured), and unlike HP or Dell they include virtual iKVM for free and a full license is only $30. Pricing on the others can easily add +$200 for what should be included functionality even if they've got a spiffier coat of paint, which is significant on a low end server device. The 5018Ds are not fanless, but running they drop down to a load speed that I can't hear over anything else I've got (like a POE switch, all of which except the tiniest models have fans). HP's Proliants in contrast are jet engines all the time even when they're pulling <50W. I'm not running a total silent environment and preferred to make a rack closet instead well away from any living areas, but I don't want loud noise and SMs have been fine. Of course, one can just replace the fans with 40mm Notcuas or the like too if desired. They've got Xeon processors that have a bit more oomph than the Atoms or Celerons and also means ECC memory, since again I really want to be able to rely on gateways to a reasonable degree (this has already identified one bad memory module out of the dozen I had around). I install OPNsense on ZFS on a cheap small decent NVMe drive (PNY CS1030 250GB is around $35) which still means fast boot and no concerns about all the logging or the like I could desire. I have one single site that is doing much heavier 10G+ routing and usage that I also wanted to mess with more intensive SDN and security with. For that last year I ended up picking the much beefier and much more expensive EPYC Embedded based 5019D-FTN4 and putting a Mellanox card in it. It's also extremely quiet and has been really impressive, but that's stupid overkill right now. Also, EPYC Embedded is currently still based off of Gen1, there was no Zen3 update due to not having low TDP given the way the chiplets were upgraded vs the IO chip. I expect Zen4 next year will see an upgraded Embedded platform that will essentially be a 3 gen leap forward, so at this point not the best time anwyay. There is no perfect solution IMO. Though probably nothing that'd throw off the typical HNer, OPNsense does have its warts, rough edges and missing bits (no Webauthn so no security keys for login for example). It's based off of a FreeBSD variant (soon to be directly off FreeBSD) with all that comes with that for better or worse. Like, OPNsense does have a user space plugin option for WireGuard (along with ZeroTier and so on), but WG has not yet made it to the FreeBSD kernel which in some situations could be an issue (countered by raw CPU in my case). But it's powerful, well maintained, overall fairly user friendly, has pretty solid documentation and getting started guides, and a nice community with a good mix of developers and some companies behind it. The company Deciso for example does offer a paid business edition and paid support options if it's desired. It does have DNS Blacklist options ala PiHole, stats/telemetry/IDS/IPS via built-in and 3rd party offerings like Sensei, etc. There are plugins for Let's Encrypt, FreeRADIUS and other handy functionality. Someone who is very familiar with Linux might find VyOS more worth looking at but with my background I found OPNsense reasonably pleasant to get into. The decision tree here does also depend on how much network functionality you want to have into your gateway/routing system vs how much to stick on a separate server elsewhere (maybe virtualized or as a part of a NAS). Gateways can be very minimal or can handle damn near everything on the network. There are straight forward tradeoffs there in terms of failure modes and complexity.

gerbilly 5y ago

I've just finished a network overhaul as well. I used similar techniques, except I preferred not to use any external DNS server¹, so I hosted a recursive DNS server (bind9) on a small fanless server (apu2d4). The DNS server can also block ads, similar to how pihole does it. 1. Why is everyone trusting cloudflare to centralize all their DNS queries? Even Firefox is migrating all their browsers to use them.

SamuelAdams 5y ago
Personally I’m using cloud flare because there’s not a lot of options. The only other major one most tech enthusiasts know about is 8.8.8.8 and that’s owned by Google. It would be good if there were more players in this space. A nonprofit might help here too, like Mozilla. Maybe they can run a public DNS server.
- tallanvor 5y ago
  quad9 is run by a foundation. Most of it's donations come from corporate sponsors, but it's definitely a better option.
- justusthane 5y ago
  I think you might be kind of missing the point here—if you run your own recursive resolver, e.g. unbound, you’re not relying on any public DNS server.

rixrax 5y ago

I never thought I would use Docker for 'production' in home network, but I recently completed a transition from three separate x64 servers to a single Raspberry Pi 4 [mostly] provisioned using Docker. Energy use-wise I went from maybe 90W to 2.9W. And to a single old-skool shell script that deploys and sets-up everything on a freshly installed Raspbian OS. In addition to the services mentioned in the article, one that I recently added to my home network is Stratum-1 NTP server that gets it's time via Pi GPIO GPS module. Whether or not this has any real positive privacy or security implications, I do like the idea that I have significantly reduced outbound traffic to/from port 123 from the devices in the home network. Interestingly most devices are happy with NTP server provided via DHCP, but notably Apple and Sonos products always want to go to time.apple.com and Sonos' alternative (and I haven't yet have had heart to setup split DNS to try and redirect them to local NTP anyways...). edit: typos

OrvalWintermute 5y ago
> I recently added to the home network is Stratum-1 NTP server that gets it's time via Pi GPIO GPS module. Whether this has any real positive privacy or security implications, I do like the idea that I have significantly reduced outbound traffic to/from port 123 from the devices in home network I like the way you think, and was thinking about some of the same. Could you share what solution you went with in terms of hardware? I am interested in using better than commodity time that could serve both NTP and precision time as well. The benefits for better time, and local time are fairly impacting if you are doing anything with time sensitive computing (PKI, performance measurement, and audio are interests of mine).
- rixrax 5y ago
  I used this board from Uputronics[0] with optionally included active GPS antenna. Mostly because they were readily available, and there were few guides online for setting ntpd up on Pi using Uputronics board. As a word of caution - it looks like most guides that I came across are outdated, or indeed have never produced a reliably (or at all) working NTPd setup. Instructions that lead me to the 'right' path is linked here[1]. I am still in the process of tuning GPS offset in my setup. But all in all, it's been a great learning experience into complexities of the protocol I've mostly taken for granted for a very long time. [0] https://store.uputronics.com/index.php?route=product/product&product_id=81 https://store.uputronics.com/index.php?route=product/product... [1] https://www.philrandal.co.uk/blog/archives/2019/04/entry_213.html https://www.philrandal.co.uk/blog/archives/2019/04/entry_213...

OrvalWintermute 5y ago

This doesn't strike me as particularly secure, private, or overengineered considering his position and level of access.

goodpoint 5y ago
overengineered for sure. Secure and private - no.

djanogo 5y ago

Running something similar with UDM on HP Prodesk G2 which I bought for $50. Ubuntu server with HomeAssistant VM and Portainer. Uses about 7w, CPU temp 32C. I was running Pi4 before but felt it was WAY overrated for this purpose, you can buy a used micro computers for $150 which come with SSD, 8GB RAM and are multiple times faster than Pi4.

BrandoElFollito 5y ago

If someone is interested, some comments on my home setup (about 4-5 years of continuous tinkering) I have FTTH (France) and the first thing I did is to replace the ISP-provided router. They are really poor quality. I bought a Ubiquity ER-4 and, retroactively, I should have bought a mini fanless PC and run Debian on it. Why Deian and not OpenSense or something like that? Because I have control over the very limited services I will use on the router, namely routing, firewalling and dnsmasq for DHCP and DNS. It is difficult to keep up with the security / functionality otherwise. Teh worst is a closed archaic system such as the one on the Ubiquity ER-4. A step up is something more dynamic such as OpenSense or similar. If you have the will to learn a bit of Linux Debian is the cleanest solution. Again - this is really for a very small set of service you will almost never touch. Behind the ER-4 there is a switch that powers a few end user PCs, a PoE port for a Ubiquity Unifi WiFi access point and a server The Unifi AP is fantastic. This is such an extreme upgrade vs the ISP provided one that I cannot imaging going back. Really invest in a higher-range consumer AP. In my apartment I used to have along the rooms (more or less linearly) 4, then 2 then 1 and 0-1 "wifi bars" (roughly the quality of reception). I now have, for the same emission power, 4 4 4 3 Then the server. There are two basic parts on it: the "services" part and the "home automation" part. The services are things (web sites, MQTT, ...) that are run exclusively from Docker. I tested the recovery of the whole system from only the backups (of the configuration (docker-compose.yaml) and the non-volatile volumes) and it took me 1 hour from "I do not have any documentation, nor ISO" to "the lights work again"). Docker is a life saver here. Backups with Borg. The second part on the server is home automation: a docker container for Home Assistant and a USB dongle for Zigbee. I then have Zigbee and Wi-Fi devices all over the appartment to get rid of fixed wall switches. Plus automation. What I would have done differently? A mini PC instead of the ER-4, two electrical wires in the wall (phase and neutral). What I will do in the future? A redundant setup for home automation (this is not simple at all). I would love to get some feedback for those who are more experienced or went though things they would do differently, or again.

willis936 5y ago
I have a pretty similar hardware setup but I went for a used Ruckus R610 WAP. With a PoE injector and shipping I spent around 200 USD. I'm really happy with it. My current goals are proper IPv6 support in case the day comes where my ISP will no longer give me an IPv4 address. The setup is currently close to perfect, but my DDNS service (google domains) isn't letting me make dynamic AAAA records for some reason. I'm in talks with support right now. I also have not successfully made ip6tables entries for rerouting IPv6 DNS traffic to my local DNS servers. When I have made entries it totally breaks IPv6. I too would prefer an x86 box for a router, but OTOH purpose-built routers are much more energy efficient. Heat is bad for reliability and more power means less run time on a UPS. My next home network purchases will be an all-flash TrueNAS box that is backed up to Backblaze. I want to get off privacy-devoid cloud services.

Johnny555 5y ago

Does anyone make an AP that allows unlimited (or at least a lot) of SSID's tied to separate VLANs? My Unifi's only support 4, so I have my private network (for my own laptops, etc), guest network, camera network and IoT network. But I'd love to be able to separate IoT devices across multiple networks, so I could have one for smart switches, one for TV/media players, etc.

rand49an 5y ago
All enterprise gear should allow you to have multiple SSIDs on different or dynamic VLANs. You won't have that issue on Meraki, Ruckus, Cisco or Aruba gear. It's worth noting though that more than 3 SSIDs brings with it radio overhead which can degrade performance of your wireless network.
- Johnny555 5y ago
  I'm fine with losing Wifi performance if I can improve my home network security. When I'm sitting at my desk, my laptop is hardwired, and my only 4K TV is hardwired, so I don't have any great bandwidth needs, and even with 4 SSID's broadcast, I'm maxing out my 500mbit internet connection, so I even if I lost half that to overhead, I'd be ok with that. I've also thought about just setting up multiple nodes, one set for secure devices, one for insecure devices. Cisco "only" supports 16 SSID's, which is probably more than I'd even need.
EvanAnderson 5y ago
You don't want "unlimited" SSIDs-- you want a RADIUS-assigned VLAN and a single SSID. The credentials used to associate determine which VLAN the traffic dumps into. Your UniFi gear will do it. Of course, a lot of consumer-grade hardware won't do 802.1x so you end up stuck with needing a bunch of SSIDs (and wasting air-time on beacons).
- Johnny555 5y ago
  Of course, a lot of consumer-grade hardware won't do 802.1x so you end up stuck with needing a bunch of SSIDs (and wasting air-time on beacons) Yes, that's why I want more SSID's. My internal network does use 802.1x, but as you said, few devices outside of laptops support it.

LgWoodenBadger 5y ago

From a physical security perspective, I cannot recommend and emphasize enough having a fiber optic media converter between your network interface (cable modem, satellite, etc.) and your expensive and extensive network. The purpose is to isolate your expensive hardware from lightning-induced destruction. The cable modem and one of the media converters serve as "sacrificial anodes" in this setup. It ends up being much cheaper to replace just a cable modem and a single media converter instead of an entire network of devices.

walterbell 5y ago
After losing a cable modem and the connected NIC port to lightning in urban apartment, I installed a lightning suppressor / surge arrester with a replaceable gas discharge tube, on both the coax cable to the cable modem, and the ethernet cable to the modem. Since then, I've once replaced the coax-facing tube/fuse due to a surge, but the equipment has been fine. Spare fuses were back-ordered at Mouser, but arrived eventually.
mongol 5y ago
That is a great point, but what about power? What do you recommend there?
- vorpalhex 5y ago
  Surge protector (high quality, insured) or UPS.

egberts1 5y ago

wow. Just dropped in a new home gateway (Devuan 3/Debian 11) - hardened Linux (Kicksecure/Whonix) - openrc (no systemd network vulnerabilities) - DMZ - NAT + IPv6 - stealth/hidden master DNS server. - Shorewall firewall - Wireless, 4 SSIDs - Wireguard to remote VPS carrying DNS queries DMZ covers all IoT, Alexa, SmartTV, game console, cable TV; both by wireless and CAT5e. Household network covers laptop, desktop, cloud servers (Proxmox), Kerberos ticket server, file server (NFSv4, media DVR, ownCloud server. And my white lab is totally airgapped and occasionally waterfalled firewall as needed. I have a subnet that offers encrypted MAC.

Jayakumark 5y ago
Do you have the instructions somewhere to follow
- egberts1 5y ago
  I’m writing them up as I go along. Need to break them up into short blogs. Now doing transparent interception squid. And I bookmarked this comment so I can reply to you later
  - Jayakumark 5y ago
    Appreciated, thanks

lambic 5y ago

I recently installed adguard home on my network because some googling suggested that was better than pi-hole. Is there a reason you picked pi-hole?

benbalter 5y ago
PiHole is what first introduced me to the DNS sinkholing concept and was more mature when I was first researching options. AdGuard home has come a long way since then and I’m planning on giving it a closer look when I’m looking for my next project.

SavantIdiot 5y ago

His research boils down to two things: 1. Lock down DNS at all costs (and he means, all costs), going so far as to use PiHole and then ridirecting DoH/DNS to CloudFlare to scan for more mal sites. 2. Create virtual subnets at home for untrusted IoT devices

HighPlainsDrftr 5y ago
This. I'm really concerned that these IoT devices will begin hard coding their own setup and using DoH to get around any DNS restrictions. I've started hardwiring devices that I have some trust in (eg laptops, desktops, etc) on to their own VLAN. I really want to setup a second wifi network, maybe a third for Rokus, IoT and other things, and force them through a proxy. Or they just don't get access to the Internet at all. It seems silly that these companies think that can use my bandwidth, whether or not it's metered, to do as they wish.

chaganated 5y ago

Where can I buy put options on github's network security?

g8oz 5y ago

Any recommendations for a dedicated OpenWRT box for the home? Links welcome. A few 2.5 GB ports would be nice.

c7DJTLrn 5y ago

It's bizarre to me - the idea of running so many things on a home network that you need to start writing Ansible playbooks, building Docker images, spinning up Kubernetes clusters, putting a rack together. I think if you really applied some thought you could get rid of all of it and save yourself the time and effort. But I guess tech people do have a tendency to find nails for their hammer.

darkwater 5y ago
It's just a hobby, it gives you some extra nice thing you wouldn't have that easily (for one, adblocking at DNS level with PiHole) but in the end is just a hobby.
- c7DJTLrn 5y ago
  I have nothing against hobbies but it's even better if your hobby has something to show for it. I think the time saved by just manually plonking the software on a cloud VPS would allow the author to do something more challenging and interesting, instead of well... pleasuring oneself with heavy duty deployment tools.
  - kipari 5y ago
    Heavy duty deployment tooling is a hobby of its own. :-)
  - laumars 5y ago
    Reading a book doesn’t give you anything tactile to show for it. You read for the experience. Likewise for gaming, solving a crossword, nor watching TV. Different people relax in different ways.
  - Godel_unicode 5y ago
    This is pretty condescending, and also misses the major point made in TFA. Writing a quick Ansible playbook/docker compose means you can quickly standup your services on any system. Run it locally on a raspberry pi, move it to a VPS, just run one command. None of this tooling is "heavy duty" by any means. If you just apply a little thought, you'll see how this saves time and effort.

lostmsu 5y ago

I am wondering if I'd prefer to use NixOS to using Docker + Ansible + yada-yada for a similar setup. I've never used either, just a guess after reading many articles.

unixhero 5y ago

A drawing would help!

ncmncm 5y ago

Ansible looks useful, but YML is a hellscape file format. Is there a system equivalent to Ansible that uses a decent config file format?

Notanothertoo 5y ago
Terraform?
beefjerkins 5y ago
What would you consider to be a decent config file format?
- ncmncm 5y ago
  TOML seems OK.

Havoc 5y ago

Fairy similar to what I did except I used proxmox and lxc not docker compose. Since proxmox is addressable via cli you can ansible against it to deploy stuff. There is also a module for it but I skipped that. I’d suggest blocking port 53 at the firewall too. I was surprised how much stuff doesn’t go through the cloudflared tunnel. You think it’s all going through the Pihole but there is so much rogue stuff

xg15 5y ago

On the whole Caddy/TLS bit: Did I miss something or wouldn't he also have to purchase a domain for the admin interface? (And then setup pihole such that the domain is resolved to the pihole server itself inside the LAN). Of course it might be that he already had a personal domain available from some other project, that he could use. But actually buying a domain just so your raspberry pi can use https seems like the ultimate overengeneering.

thedanbob 5y ago
You're correct, but you can see he already has a personal domain hosting his blog: ben.balter.com. He probably just pointed another subdomain at the pihole e.g. pihole.balter.com. That's what I did with my piholes and other internal web apps on my home network.

aaroninsf 5y ago

De rigeur despairing comment, which will be downvoted into obscurity: That such efforts are necessary, and as the commentary shows, an active area of interest and hacking and development by so many very smart people, is an absolute condemnation on the dystopian state of the surveillance capitalism that so many of us here have helped build. It's not malicious actors that are the sole or even primary vector being blocked. It's the now-systemic misbehavior of so many of our own products and services.

selfhoster11 5y ago
I'm not sure what you're even trying to say here.
- aaroninsf 5y ago
  I'm saying that it's well worth taking a step back from the tree-level view, e.g. celebrating and being inspired by the intellectual and technical details of a smart industrious individual attempt to reasonable degree insulate themselves from the abuses of surveillance capital, and consider the forest-level view, that while few were paying much attention, our society in general, and the industry most HN readers contribute to in specific–in some cases, very very directly–is now profoundly, all but irredeemably broken, wrong, amoral, and bad, pick a word. Articles of this type and fandom and advocacy of this or that open source solution to a tiny piece of the surveillance capitalism nightmare, are in small way part of the problem; they normalize the experience well sketched here: https://den.dev/blog/user-hostile-software/ https://den.dev/blog/user-hostile-software/ But that article only approaches some sides of the problem. The political and philosophical ones, which are implicit but not blown out, are deep, real, and corrosive. The subtext, which is not so subterranean, is that readers here who are cashing a pay check accelerating total information awareness, dark patterns, and the abuse of users in exchange for calculatedly seductive or addictive services whose fine print all know no one ever reads, are culpable, and that damage is real, and the costs real. And they should do better.

noncoml 5y ago

"How I re–over-engineered my home network for privacy and security" Proceeds to send all his DNS queries to Cloudflare

KeepFlying 5y ago
Still better than sending them to his ISP (or keeping them visible to his ISP no matter the destination). Sure it's not ideal but really very few things are when it comes to home network DNS requests. I'd love suggestions for better solutions though, I'm sure there's something I haven't considered.
- noncoml 5y ago
  I am a hypocrite as I do the exact same thing, Pi-Hole + Cloudflare. It's just that your title raised my hopes for a better solution. Sorry :)
- Skunkleton 5y ago
  I intercept port 53 traffic at my router, tunnel it over to my VPS which runs an unbound instance. Overall I think this setup is better because: 1. It denies my ISP the chance to look at my DNS requests. 2. It doesn't involve yet another third party which may or may not try to monetize my dns requests. 3. It mixes my dns related traffic with that of a datacenter, which doesn't provide a compelling source of data for advertisers IMO. DoH throws a wrench in the works. Still considering my options on how I want to address it.

firecall 5y ago

Nice article, but not really over engineered. More, spent money on a bunch of gear and installed PiHole! I did the same though LOL However, in the spirit of his comments about Outsourcing to The Experts, I switched from PiHole to AdGuard Home to NextDNS. Very happy with NDNS, just wish there was a kill switch! I'm up around 50 devices on my home network, so the standard home router from the ISP, or even the fancy 'gaming routers' that look like alien spiders don't really cut it! I get terrible internet where I am in Australia, so I have to be creative: ---------- My Setup: ---------- • 3x Unifi UAPs • 1x USG Pro • 1x USW 16 POE • 2x 4G modems Connected to the USG Pro and load balanced! You can't LB with a UDM! •1x M1 Mac mini with 12TB RAID as a TMS and Content Cache! Serves around ~500GB a month to my network!

devnulll 5y ago

I've done a similar setup at home with a full Unifi setup, PiHole, DNS over TLS, Let's Encrypt certs for internal devices, multiple wifi networks and other "It's COVID and I'm bored" goodness. I've had to turn most of it off. PiHole means every single web site issue for any user on the WiFi ends up on my plate immediately. Wifi network segments mean Minecraft doesn't "Just work" for the kids and their friends. IOT networks means Airplay doesn't find devices. SDDP doesn't work, etc. Having a functional home network turns out to be more important than having a secure home network.

bacan 5y ago

This post makes no sense, at all. 1) UniFi devices just recently had a massive security vulnerability. Also not a good idea to be letting third party servers, access your home network, directly via your router 2) Instead of CloudFlare, use something like Unbound. I have mine setup to fetch directly from the Root servers. Why send your DNS queries upstream? 3) Caddy is the one decent thing suggested, however I am skeptical of the benefits of having HTTPS internally in your LAN. If an attacker is in your LAN, then it is already Game Over

dvcrn 5y ago

I went down a similar hole with a UniFi Dream Machine. udm-utilities [0] basically allows you to run all kinds of stuff on it through podman. Went from trying to teach it to mimic my ISPs router to get internet working, to banging my head against IPv6 prefix delegation, to now running adguard, homebridge and a bunch of other things across different VLANs on it. All in all, I am happy with the result, and happier that I got dragged out of my progrmaming bubble to learn proper networking with IPv6. [0]: https://github.com/boostchicken/udm-utilities https://github.com/boostchicken/udm-utilities

8fingerlouie 5y ago

Sounds like my old networking setup, except i had a special "IoT like" VLAN for my kids and their friends. No reason for malware to propagate outside the kids network :) Besides that was a DMZ and a storage VLAN, and because i'm cheap i opted for a router on a stick kinda setup with all storage being mounted across the firewall as kerberized NFSv4. Ideally i would probably have setup an L3 switch with ACLs, but i had the processing power / bandwidth on the router. These days i've taken it at step further (back). PiHole got replaced by AdGuard Home, which then in turn got replaced by NextDNS. PiHole/AdGuard Home only works on your own network, and NextDNS works everywhere. The price they're asking ($20/year) is less than the cost of the Raspberry Pi and power required to run it, and requires no maintenance on my end. Everything that previously ran in docker at home has been replaced with Cloud equivalent services, source encrypted by either rclone + encryption, or just using Cryptomator. Again, the cost of the cloud services is cheaper (about half) of what i paid in power consumption for just running the server. Then add hardware costs on top of that. As for my home network, since everything is now "out there", i've retired about half my network infrastructure (yay, less stuff that can break), and the internet connection is now the lower common denominator. I used some of the money i saved on power consumption to instead upgrade my internet connection to a 500/500. All that remains is a single Mac Mini that is powered 24/7 for the purpose of pulling our iCloud data back home so that i can make backups. I would really love if TimeMachine or 3rd party backup tools could backup that cloud data directly from the client, downloading it only once, and other than that leave it in the cloud.

SavantIdiot 5y ago

UniFi now offers L2 isolation on WLANs, so I don't think he needs a VLAN for the IoT wifi. How do you create a rule to isolate a printer? It still needs to receive requests from the network, but I don't want it probing other devices...?