3 ms·

I know, I'll probably get some flack but I love running Kubernetes at home. A combination of Flux, github and actions, and renovate makes it very easy to mainta

I know, I'll probably get some flack but I love running Kubernetes at home. A combination of Flux, github and actions, and renovate makes it very easy to maintain.

Cesura 2y ago
I agree completely. Most of the complaints I read about Kubernetes being "overkill" for homelabbing seem to stem from either wildly overestimating the resource requirements, or underestimating just how many common infrastructure concerns it handles for you out of the box (things that would otherwise still need to be implemented by the administrator in various ways, but with a less unified config syntax).
- gardnr 2y ago
  The first time I went to set up kubernetes the documentation explained what each of my three machines would do. It's different now but I still find the syntax needlessly verbose. So now instead, I have a rediculously long docker compose yml file. What are some of the common infrastructure concerns it handles for you out of the box?
- nullify88 2y ago
  The operator pattern and custom resources allows me to extend the kubernetes API to cover certificates, backups, encrypted secrets, and more in a consistent centralised way. I preferred to start with an empty cluster and to add in the functionality needed.
captn3m0 2y ago
Which k8s distribution are you running? I've considered running k3s/typhoon in the past, but never had a requirement for more than one host. Plus managing stateful infra on k8s feels like a full-time job.
- nullify88 2y ago
  I tried k3s but wasn't happy with its resource utilisation. I settled on k0s with integrated etcd, and the crun container runtime. Additionally k0s deployments can managed with k0sctl and yaml. I run a single intel n100 node running Fedora Core. Initially it was a lot of tinkering with IaC / yaml but once I had a working setup, I spend most of the time merging PRs from renovate from my phones GitHub app that keeps the software updated. Some supporting benchmarks. https://www.scitepress.org/Papers/2020/93404/93404.pdf https://www.scitepress.org/Papers/2020/93404/93404.pdf https://programming-group.com/assets/pdf/papers/2023_Lightweight-Kubernetes-Distributions.pdf https://programming-group.com/assets/pdf/papers/2023_Lightwe...
- neurostimulant 2y ago
  If the server is small (<= 4gb of RAM), I would use k3s. For larger servers I would use rke2. Keeping them up to date is pretty easy, just update the os, update rke2/k3s, then restart the cluster.
neurostimulant 2y ago
I've been using rke/rke2 and k3s for a few years now. I still use docker compose, but once the docker compose file reaches certain level of complexity, I would migrate it to rke2 or k3s (depending on the size of the server).

I know, I'll probably get some flack but I love running Kubernetes at home. A combination of Flux, github and actions, and renovate makes it very easy to maintain.

Cesura 2y ago

I agree completely. Most of the complaints I read about Kubernetes being "overkill" for homelabbing seem to stem from either wildly overestimating the resource requirements, or underestimating just how many common infrastructure concerns it handles for you out of the box (things that would otherwise still need to be implemented by the administrator in various ways, but with a less unified config syntax).

gardnr 2y ago
The first time I went to set up kubernetes the documentation explained what each of my three machines would do. It's different now but I still find the syntax needlessly verbose. So now instead, I have a rediculously long docker compose yml file. What are some of the common infrastructure concerns it handles for you out of the box?
nullify88 2y ago
The operator pattern and custom resources allows me to extend the kubernetes API to cover certificates, backups, encrypted secrets, and more in a consistent centralised way. I preferred to start with an empty cluster and to add in the functionality needed.

captn3m0 2y ago

Which k8s distribution are you running? I've considered running k3s/typhoon in the past, but never had a requirement for more than one host. Plus managing stateful infra on k8s feels like a full-time job.

nullify88 2y ago
I tried k3s but wasn't happy with its resource utilisation. I settled on k0s with integrated etcd, and the crun container runtime. Additionally k0s deployments can managed with k0sctl and yaml. I run a single intel n100 node running Fedora Core. Initially it was a lot of tinkering with IaC / yaml but once I had a working setup, I spend most of the time merging PRs from renovate from my phones GitHub app that keeps the software updated. Some supporting benchmarks. https://www.scitepress.org/Papers/2020/93404/93404.pdf https://www.scitepress.org/Papers/2020/93404/93404.pdf https://programming-group.com/assets/pdf/papers/2023_Lightweight-Kubernetes-Distributions.pdf https://programming-group.com/assets/pdf/papers/2023_Lightwe...
neurostimulant 2y ago
If the server is small (<= 4gb of RAM), I would use k3s. For larger servers I would use rke2. Keeping them up to date is pretty easy, just update the os, update rke2/k3s, then restart the cluster.

neurostimulant 2y ago

I've been using rke/rke2 and k3s for a few years now. I still use docker compose, but once the docker compose file reaches certain level of complexity, I would migrate it to rke2 or k3s (depending on the size of the server).