5 ms·

> I ended up with 3 distinct boxes: router/gateway (OPNsense), VM system, and NAS system. This was the first lesson I learned doing homelabbing. Put your neces

by SxC97 5y ago

> I ended up with 3 distinct boxes: router/gateway (OPNsense), VM system, and NAS system. This was the first lesson I learned doing homelabbing. Put your necessary networking stuff on it's own system that won't be affected by your "hobby boxes" going down :)

ziml77 5y ago
I've seen people talk about virtualizing their router and always thought that was a really bad idea. It feels good not having a ton of single-purpose boxes. But it doesn't take much playing around with them to realize the problem with putting all your important stuff on the same machine: it all goes down together. I currently have my stuff split out into one PCEngines box for OPNSense, one Lenovo NUC for HomeAssistant, and one tower for TrueNAS and low importance virtualzation.
- xoa 5y ago
  >I've seen people talk about virtualizing their router and always thought that was a really bad idea. It feels good not having a ton of single-purpose boxes. Yeah, there is definitely a balance to be struck. Too many single purpose boxes is very inefficient and also results in its own set of problems in terms of backups and restores. >But it doesn't take much playing around with them to realize the problem with putting all your important stuff on the same machine: it all goes down together. For me at least even that wasn't the biggest problem. It was more the extra difficulty with getting stuff back up again, basically things turning into classic spaghetti code for IRL metal. If one thing took down the other stuff that'd be annoying, but if merely booting back up again fixed everything I'd have felt less compulsion. But if one doesn't have a functional network/internet (particularly in more rural areas like mine where there is effectively zero cellular coverage so no emergency fallback there) it can be a lot harder to recover from a real fubar. And if there is some level of circular dependencies it was easily possible to end up in situation where order is critically important just for startup let alone recovery, and things like backups get fragmented. If the NAS is virtualized yet other VMs live on the NAS, the NAS VM must come up first or other stuff just won't work, including the network if the router is virtualized too on the same system. And where does the hypervisor and NAS VM itself get backed up to? How do those tie into remote backup automation? If things go pearshaped, how does bootstrapping off an offsite/cloud restore go? What's downtime look like? It's definitely possible to set up orchestration to do all this, or HA clusters with high enough reliability it wouldn't be a limiting factor or a bunch of other solutions. But for homelab/SOHO/SMB too much for my comfort. On the other hand I'm fine with something like HomeAssistant being virtualized and consider that more reliable. The VM for it runs on the hypervisor box, the data is stored on the NAS box, and the OPNsense box handles the routing. All 3 can stand alone. If any of them go bad, it's clear how I bring each one back up, and I can lean on the the other two. If the whole place burned down, dealing with insurance aside it's still clear how I'd bring the whole thing back up on new hardware at a new location once I had it. And experimentation done in one place isn't likely to leave me in a tricky position to get back to working. If I was really depending on HomeAssistant or the like for critical home functionality maybe I'd give it its own box too, but even then I still feel like the isolation, central backup, protection against data corruption and so on provided by having it virtualized is better. Also easier to rollback the whole thing. Have to say though also feels amazing that we have so many powerful awesome options all on FOSS and standard hardware now. Can do really great stuff on a budget that not that long ago would have been the domain of enterprise.

by SxC97 5y ago

ziml77 5y ago

I've seen people talk about virtualizing their router and always thought that was a really bad idea. It feels good not having a ton of single-purpose boxes. But it doesn't take much playing around with them to realize the problem with putting all your important stuff on the same machine: it all goes down together. I currently have my stuff split out into one PCEngines box for OPNSense, one Lenovo NUC for HomeAssistant, and one tower for TrueNAS and low importance virtualzation.

xoa 5y ago
>I've seen people talk about virtualizing their router and always thought that was a really bad idea. It feels good not having a ton of single-purpose boxes. Yeah, there is definitely a balance to be struck. Too many single purpose boxes is very inefficient and also results in its own set of problems in terms of backups and restores. >But it doesn't take much playing around with them to realize the problem with putting all your important stuff on the same machine: it all goes down together. For me at least even that wasn't the biggest problem. It was more the extra difficulty with getting stuff back up again, basically things turning into classic spaghetti code for IRL metal. If one thing took down the other stuff that'd be annoying, but if merely booting back up again fixed everything I'd have felt less compulsion. But if one doesn't have a functional network/internet (particularly in more rural areas like mine where there is effectively zero cellular coverage so no emergency fallback there) it can be a lot harder to recover from a real fubar. And if there is some level of circular dependencies it was easily possible to end up in situation where order is critically important just for startup let alone recovery, and things like backups get fragmented. If the NAS is virtualized yet other VMs live on the NAS, the NAS VM must come up first or other stuff just won't work, including the network if the router is virtualized too on the same system. And where does the hypervisor and NAS VM itself get backed up to? How do those tie into remote backup automation? If things go pearshaped, how does bootstrapping off an offsite/cloud restore go? What's downtime look like? It's definitely possible to set up orchestration to do all this, or HA clusters with high enough reliability it wouldn't be a limiting factor or a bunch of other solutions. But for homelab/SOHO/SMB too much for my comfort. On the other hand I'm fine with something like HomeAssistant being virtualized and consider that more reliable. The VM for it runs on the hypervisor box, the data is stored on the NAS box, and the OPNsense box handles the routing. All 3 can stand alone. If any of them go bad, it's clear how I bring each one back up, and I can lean on the the other two. If the whole place burned down, dealing with insurance aside it's still clear how I'd bring the whole thing back up on new hardware at a new location once I had it. And experimentation done in one place isn't likely to leave me in a tricky position to get back to working. If I was really depending on HomeAssistant or the like for critical home functionality maybe I'd give it its own box too, but even then I still feel like the isolation, central backup, protection against data corruption and so on provided by having it virtualized is better. Also easier to rollback the whole thing. Have to say though also feels amazing that we have so many powerful awesome options all on FOSS and standard hardware now. Can do really great stuff on a budget that not that long ago would have been the domain of enterprise.