3 ms·
I use proxmox, which is more or less a VM and workflow manager on top of KVM. The overhead on something like an RPi would be ridiculous, but on modern x86 hard
by xxpor 4y ago
I use proxmox, which is more or less a VM and workflow manager on top of KVM.
The overhead on something like an RPi would be ridiculous, but on modern x86 hardware with an IOMMU (VT-d in Intel speak, AMD-Vi for AMD), the overhead of passing through HW is, for homelab purposes, essentially 0. A lot more expensive, but the organization and extensibility is well worth it.
I have anything that I expose directly to the internet on a separate VM from my "internal" services. If I were super paranoid, I'd expose them to separate VLANs, and then use my FW to control network traffic. The Intel 82599 can enforce different vlans on different VFs with SR-IOV.
I have a VM that runs flatcar for docker for things that are too hard to set up otherwise, but I vastly prefer NixOS for most things.
- float4 4y ago> If I were super paranoid, I'd expose them to separate VLANs, and then use my FW to control network traffic This is exactly what I did initially, but it was indeed a bit of a pain to manage. Eventually I went with something in between, by first compartmentalizing services and then putting them in separate VMs with separate VLANs: 0. Router / FW. 1. WireGuard / reverse proxy. 2. Personal, e.g. file storage, backups. 3. Hosting. My personal site is reverse proxied through Cloudflare and only their IP ranges are whitelisted. 4. Compute, i.e. stuff I want to compile / develop / run on my server. Handy if I want to run a heavy simulation overnight or need more disk space / RAM / CPU power than my M1 MB Air has available. 5. Services. This runs many small tools / services that don't need access to my RAID pool or anything like that. If this gets infected I wouldn't really care. 6. VPN. This VM can only access the internet through a VPN. Doesn't have anything installed ATM, but has been used in the past for urlwatch and torrenting. 7. Test. This is where I try out new software before actually installing it on the correct VM. Once I've concluded testing I rollback this VM to a clean install. It takes a weekend to install Proxmox and set up the VMs / VLANs, but after that it easy to use.
- lostlogin 4y agoAnother option is the free tier of ESXi. It works well, but having tested Proxmox recently, I really liked it.
- blibble 4y agoit seems to be almost impossible to find a machine that is both low power and also supports SR-IOV ARI (more than 8 VFs) and the best reason to use SR-IOV with networking is you completely avoid the awfulness that is the Linux bridging/firewalling stack
- kfrzcode 4y agoAny advice on how to add n Raspberry Pi 4b to a Proxmox setup? Maybe just as bare docker nodes? I've used them for Nomad and k3s before, but Proxmox seems a bit heavy resource-wise in comparison.
- xxpor 4y agoDoes an RPi 4b even have virtualization extensions? I'm wondering even if it can even run KVM.