3 ms·
Been homelabbing in some capacity for decades, current iteration ~5y in now. Here are some pointers from me to you, with the assumption that you will end up wi
by baobun 3y ago
Been homelabbing in some capacity for decades, current iteration ~5y in now.
Here are some pointers from me to you, with the assumption that you will end up with some form of personal production workloads and that extended unplanned downtime, security breaches and data-loss would be stress-inducing experiences you want to avoid.
- For any future hardware you add, get minimum 2 of everything. You never know when or why the extra will prove invaluable.
- Get a managed L2 PoE switch with more ports than you think you need. Used Brocade/Ruckus gear can be found on eBay, for example. Until you do, at your current scale you can get away with several cheaper smaller unmanaged switches. But I'd be considering getting more serious gear when start growing out of 10-12 Ethernet ports if you aren't already sick of cables and wall-warts at that point.
- The STH forum is an amazing resource. Identify and scourge megathreads relevant to you.
- Segregate your networks. Don't run your servers on the same network where you have your WiFi AP and user clients. Ideally, your servers won't even have a default gateway and be firewalled to only allow internal traffic even for the outgoing. You will set up not only reverse proxies for the incoming (Caddy will be the easiest to get started with if it's all the same to you) but also for the outgoing (squid still seems to be the sane default for HTTP?). You can still proxy HTTPS via an HTTP CONNECT proxy without having to care about TLS, certs, or MitMing.
- One piece to the above would be setting up a "bastion host" - the gateway and firewall between your labnet and the world. You want something with minimum 2 NICs. My personal experience with USB NICs has not been great. I strongly advice you to consider this use-case for the next piece of hardware you get alongside the switch. A cheap SBC (get minimum 2!) should be fine.
- Wireguard
- Use configuration management and resist the urge to manually configure stuff by SSHing and editing files. You want to be able to reproduce it and keep track of changes you made. Ansible is popular but there are many others - pick whatever feels smoother for you.
- Backups: do them. Learn about the 3-2-1 rule and apply it. You could let one of your Rpis with an HDD be a dedicated backup-sink.
- Virtualization: I'd say you shouldn't bother with this at all for now, unless learning virtualization is a goal in itself. Hypervisors like Proxmox make a lot of sense if you have 1 or 2 huge hosts. You have a larger number of smaller hosts already. Makes more sense to scale horizontally and use containers (Docker/LXC) to separate workloads within one host. If you get something more beefy than the Mac Mini down the line, it can start making sense to look into, though.
- Just Do It. You don't need any of the above to start iterating and prototyping today. Configuration management will make it easier to fearlessly set up and tear down your setup.
- baobun 3y agoAs for my own main setup: - 48 port Brocade PoE switch - 2x "Bastion": N100-based Mini-PC with 4 NICs. Qemu+libvirt, running several VMs for proxies, firewalls, VPN gateways and such. These are the only hosts connected directly to the network. The 2 are identical and made redundant using VRRP. - 9x Pi 4B-equivalent ARM SBCs: 3x consul + 3x vault + 3x nomad server. I'm considering virtualizing these onto 3x of something similar to the bastion host. - 3x x86 server boards with ECC RAM, and various drives, virtualized: Glusterfs servers, database servers, nomad clients. Distributed filesystem was probably premature here and I regret glusterfs but it's still chugging along. - A few other hosts: Nomad clients for workloads not suitable on the above for whatever reason. - x86 miniPC with 2x16GB HDDs as dedicated backup sink (zrepl) All Debian base. Running most of my own needs on this platform. I also have some home-automation stuff with HomeAssistant and Zigbee devices etc but that's a completely different setup and network.