6 ms·

Why would you do this for simple home apps? k8s is complete overkill. At home, there is little reason to drag in all that complexity to host a blog or whatever.

Why would you do this for simple home apps? k8s is complete overkill. At home, there is little reason to drag in all that complexity to host a blog or whatever. Kubernetes drags in a tremendous amount of complexity, background knowledge, and strange constraints, even on small installations. I really do not see its benefit for small apps and projects. Are you hosting your blog? You don't need k8s. Are you running a small app server at home? Nope, don't need it. Are you running an auto-scaling app with many hundreds or thousands of worker instances, and need to support non-disruptive roll-in upgrades across multiple clouds and have many services which need to scale independently? Maybe you need k8s, but only in some circumstances. For the record, I run many k8s clusters across several clouds, using 1,000 - 50,000 cores at any one time, so I've dealt with quite a bit of k8s complexity, and I'm still on the fence whether it's the right answer for us, but it's allowed us to standardize our software to k8s and only worry about getting that running well in each deployment, which put the cross-cloud work on the infrastructure teams versus the software development teams. The price you pay for this is that you still need to do non-k8s stuff if you need some cloud specific resources more than simple compute and routing, so you have both k8s and cloud specific code.

Bombthecat 6y ago
Learning?
- oppositelock 6y ago
  That is a fine reason!
brooksgarrett 6y ago
I'll second you. I full time admin our K8s infrastructure and clusters. It's a great solution for our production workloads and I've bootstrapped RancherOS/k3s a handful of times to look at using it at home but K8s is purpose built for cloud. I can get all the same benefits with significantly reduced complexity with Docker + Systemd at home.
- MuffinFlavored 6y ago
  what are some usecases where you need systemd alongside docker containers?
  - elbear 6y ago
    systemd can restart your container if it crashes. It can also start it if you reboot your system.
    - MuffinFlavored 6y ago
      can you set a restart policy with just Docker?
      - elbear 6y ago
        I don't know. But if you can, that still leaves you the second use case: starting your container on reboot. Can you have Docker do that too?
g-clef 6y ago
I'll bite. One thing I found in running my home lab was that I kept having to burn it to the ground & rebuild it regularly anyway. For example, dist-upgrade never really works: every time I try I just end up wasting a couple days wrestling with it and then giving up and rebuilding the machine from scratch. Even if I just assumed that I'd have to build from scratch every time, differing app and library versions meant that I couldn't count on a new build being a simple, clean install. So going with the regular "run things as a daemon on a server" model wasn't actually saving me that much time. Basically I could to do one of two things: 1. keep using purpose-configured machines and spend a bunch of time writing ansible scripts to automatically re-create them when everything goes pear-shaped and then re-write the scripts when a new version changes stuff. 2. container-ize all my tasks, and make everything else vanilla and effectively disposable. I have to spend some initial time to rewrite my stuff in container-speak, but that's a one-time cost. Presented like that, option 2 looked like a better option. When a machine has a problem or needs to be rebuilt, I build it with the completely vanilla setup (ubuntu lts, conjure-up k8s) and push my k8s jobs and pods up to it. That's 2 hours instead of a day and a half. (Yes, in theory I could docker-ize everything and run docker-swarm, but it's a small step from there to k8s, and conjure-up makes installing k8s fairly straightforward.) Frankly, I have a family now, and fucking with fiddly settings and library dependencies isn't fun anymore. I'd rather spend that time doing stuff. k8s lets me divorce my hobby work from the infrastructure, and I like that.
- hddherman 6y ago
  I think you could achieve something similar but by just using simple standalone containers that you start with Podman or Docker. Once you have the OS up and running, then all you need to do is push the configuration to the server (I just rsync systemd service files) and start the services. Persistent data is stored elsewhere anyway, so moving a service is just a matter of copying a directory over to another machine and ensuring that the permissions are correct. k8s is probably great for many things, but as time goes on I really appreciate the ability to debug things effectively and by keeping my homelab setup as simple as possible I avoid the complexity of whatever advanced solution that is out there.
  - 6y ago

Bombthecat 6y ago

Learning?

oppositelock 6y ago
That is a fine reason!

brooksgarrett 6y ago

I'll second you. I full time admin our K8s infrastructure and clusters. It's a great solution for our production workloads and I've bootstrapped RancherOS/k3s a handful of times to look at using it at home but K8s is purpose built for cloud. I can get all the same benefits with significantly reduced complexity with Docker + Systemd at home.

MuffinFlavored 6y ago
what are some usecases where you need systemd alongside docker containers?
- elbear 6y ago
  systemd can restart your container if it crashes. It can also start it if you reboot your system.
  - MuffinFlavored 6y ago
    can you set a restart policy with just Docker?
    - elbear 6y ago
      I don't know. But if you can, that still leaves you the second use case: starting your container on reboot. Can you have Docker do that too?

g-clef 6y ago

I'll bite. One thing I found in running my home lab was that I kept having to burn it to the ground & rebuild it regularly anyway. For example, dist-upgrade never really works: every time I try I just end up wasting a couple days wrestling with it and then giving up and rebuilding the machine from scratch. Even if I just assumed that I'd have to build from scratch every time, differing app and library versions meant that I couldn't count on a new build being a simple, clean install. So going with the regular "run things as a daemon on a server" model wasn't actually saving me that much time. Basically I could to do one of two things: 1. keep using purpose-configured machines and spend a bunch of time writing ansible scripts to automatically re-create them when everything goes pear-shaped and then re-write the scripts when a new version changes stuff. 2. container-ize all my tasks, and make everything else vanilla and effectively disposable. I have to spend some initial time to rewrite my stuff in container-speak, but that's a one-time cost. Presented like that, option 2 looked like a better option. When a machine has a problem or needs to be rebuilt, I build it with the completely vanilla setup (ubuntu lts, conjure-up k8s) and push my k8s jobs and pods up to it. That's 2 hours instead of a day and a half. (Yes, in theory I could docker-ize everything and run docker-swarm, but it's a small step from there to k8s, and conjure-up makes installing k8s fairly straightforward.) Frankly, I have a family now, and fucking with fiddly settings and library dependencies isn't fun anymore. I'd rather spend that time doing stuff. k8s lets me divorce my hobby work from the infrastructure, and I like that.

hddherman 6y ago
I think you could achieve something similar but by just using simple standalone containers that you start with Podman or Docker. Once you have the OS up and running, then all you need to do is push the configuration to the server (I just rsync systemd service files) and start the services. Persistent data is stored elsewhere anyway, so moving a service is just a matter of copying a directory over to another machine and ensuring that the permissions are correct. k8s is probably great for many things, but as time goes on I really appreciate the ability to debug things effectively and by keeping my homelab setup as simple as possible I avoid the complexity of whatever advanced solution that is out there.
- 6y ago