4 ms·

> Practically, it is foolishness, for what you save in money you lose in time and sanity. Kubernetes gets a lot of side eyes in the self-hosted community. That

> Practically, it is foolishness, for what you save in money you lose in time and sanity. Kubernetes gets a lot of side eyes in the self-hosted community. That's all of self hosting though. So why not go all in? I've got 3 dell r720XDs running nixos with k3s in multi master mode. It runs rook/ceph for storage, and I've got like 12 hard drives in various sizes. My favorite party trick is yoinking a random hard drive out of the cluster while streaming videos. Does not care. Plug it back in and it's like nothing happened. I've still got tons of room and I keep finding useful things to host on it.

kayson 2y ago
Plenty of people use k8s or k3s for self hosting. But for most, the added complexity doesn't buy enough for the trade-off to be worth it. Keep in mind most people have a single node, so docker does everything they need. Personally, even with a 4 node setup (of tiny desktops; the hardware you have would easily cost me $200/mo in power bills), I use docker swarm. Old and unloved, but does everything I need for multi node deployment and orchestration with only a sliver more complexity than vanilla docker.
- from-nibly 2y ago
  Yeah don't ask me about my power bill, It's definitely in the vanity realm. I have cheap power where I live so it's not anywhere near $200. Still too high though. One day I'll get solar to offset it.
Cyph0n 2y ago
I just use NixOS as a VM and run services as containers directly. Self-plug: I wrote a tool that makes it easy to run Docker Compose projects on NixOS [1]. This way, I get the advantages of NixOS config, while also being able to run arbitrary applications that might not be available on nixpkgs. As far as storage goes, I just use ZFS on the hypervisor (Proxmox) and expose that over NFS locally. [1] https://github.com/aksiksi/compose2nix https://github.com/aksiksi/compose2nix
jamwil 2y ago
For a homelab k8s is way overkill. I do it because self-hosting is 1 part utility to 2 parts education.
- pl4nty 2y ago
  it's become much easier in the last few years. Talos + Flux + Renovate can be built in a day [0], and simplifies storage/backups/patching even for single-node clusters. There's also a great community, with services like kubesearch [1] providing templates for tons of apps [0] https://github.com/onedr0p/cluster-template https://github.com/onedr0p/cluster-template [1] https://kubesearch.dev https://kubesearch.dev

kayson 2y ago

Plenty of people use k8s or k3s for self hosting. But for most, the added complexity doesn't buy enough for the trade-off to be worth it. Keep in mind most people have a single node, so docker does everything they need. Personally, even with a 4 node setup (of tiny desktops; the hardware you have would easily cost me $200/mo in power bills), I use docker swarm. Old and unloved, but does everything I need for multi node deployment and orchestration with only a sliver more complexity than vanilla docker.

from-nibly 2y ago
Yeah don't ask me about my power bill, It's definitely in the vanity realm. I have cheap power where I live so it's not anywhere near $200. Still too high though. One day I'll get solar to offset it.

Cyph0n 2y ago

I just use NixOS as a VM and run services as containers directly. Self-plug: I wrote a tool that makes it easy to run Docker Compose projects on NixOS [1]. This way, I get the advantages of NixOS config, while also being able to run arbitrary applications that might not be available on nixpkgs. As far as storage goes, I just use ZFS on the hypervisor (Proxmox) and expose that over NFS locally. [1] https://github.com/aksiksi/compose2nix https://github.com/aksiksi/compose2nix

jamwil 2y ago

For a homelab k8s is way overkill. I do it because self-hosting is 1 part utility to 2 parts education.

pl4nty 2y ago
it's become much easier in the last few years. Talos + Flux + Renovate can be built in a day [0], and simplifies storage/backups/patching even for single-node clusters. There's also a great community, with services like kubesearch [1] providing templates for tons of apps [0] https://github.com/onedr0p/cluster-template https://github.com/onedr0p/cluster-template [1] https://kubesearch.dev https://kubesearch.dev