4 ms·

> I moved to docker swarm and love it. It's so much easier, straight forward, automatic ingress network and failover were all working out of the box. I'll stay

> I moved to docker swarm and love it. It's so much easier, straight forward, automatic ingress network and failover were all working out of the box. I'll stay with swarm for now. I've had decent luck in the past with the K3s distribution, which is a bit cut down (but certified) Kubernetes: https://k3s.io/ https://k3s.io/ Others might also mention K0s, MicroK8s or others - there's lots of options there. It also integrates nicely with Portainer (aside from occasional Traefik ingress weirdness sometimes), which I already use for Swarm and would suggest to anyone that wants a nice web based UI for either of the technologies: https://www.portainer.io/ https://www.portainer.io/ But even so, I still run Docker Swarm for most of my private stuff as well and it's a breeze. For my needs, it has just the right amount of abstractions: stacks with services and replicas that use networks and can have some storage in the form of volumes or bind mounts. Configuration in the form of environment variables and/or mounted files (or secrets), some deployment constraints and dependencies sometimes, some health checks and restart policies, as well as resource limits. If I need a mail server, then I just have a container that binds to the ports (even low port numbers) that I need and configure it. If I need a web server, then I can just run Apache/Nginx/Caddy and use more or less 1:1 configuration files that I'd use when setting up either outside of containers, but with the added benefit of being able to refer to other apps by their service names (or aliases, if they have underscores in the names, which sometimes isn't liked). At a certain scale, it's dead simple to use - no need for PVs and PVCs, no need for Ingress and Service abstractions, or lots and lots of templating that Helm charts would have (although those are nice in other ways). Unfortunately, Swarm isn't awfully good for job prospects, which I have to admit. Then again, one can sort of say the same about Hashicorp Nomad, yet it's also alive and kicking.

mulmen 3y ago
Kubernetes should be called Kafka but the name was already taken. It feels deliberately complicated. I’m honestly curious about Kubernetes and I want to learn more but wow, it really is hard to grok. What does it mean that k3s is “certified”? Should I care? What’s the tradeoff? When you mentioned Traefik my eyes glazed over. Is there a picture of how all this fits together? One not created by a consultant selling their services?
- hosh 3y ago
  It isn’t deliberately complicated, and once you know why it is designed the way it is, it becomes very capable. The biggest thing to understand about Kubernetes is that you cannot understand it as a normal piece of software. You don’t directly control it, but rather, define the conditions under which a collection of automations respond to things going on. This is how it self heals. The whole system resembles more of a garden, where you are providing the conditions in which the plant grows, but you are not directly causing the plant to grow. For example, if you understand a pod, then the next step is understanding how to keep a pod running, how to detect failures, what go do if it does fail, and how to change out one set of pods for the next. This would be a deployment. You build your understanding by understanding more independent automations. For example, after understanding a deployment, you can look at how horizontal pod autoscalers work. An HPA knows nothing about a deployment and its internal state, yet will inform the deployment how many pods it should be running based on the metrics it sets. The HPA and the deployment scheduler otherwise work independently, and you can indeed write something that functions as an alternate HPA, and Kubernetes will do it. There’s also an addon for cluster autoscaling. The cluster autoscaling knows nothing of deployments or HPA. It only checks pods that are being requested, but cannot be scheduled due to constraints. It then add nodes so that pods can be scheduled. These three different processes operate independently, and don’t share state. They each watch for a limited set of signals and take limited set of actions. If any one of these processes goes down, the system can still limp along. Together, they create a much more complex behavior out of much simpler behaviors.
  - mulmen 3y ago
    > It isn’t deliberately complicated, and once you know why it is designed the way it is, it becomes very capable. I’m sure it isn’t and I changed the emphasis to make that line more about the slope of the learning curve. Thanks for this explanation. The inability to grok the internals is really hanging me up. I’ll embrace the simpler “distributions” as a way to understand the higher level constructs. Does it also make sense to ignore the fact that this is built on Linux’s containers and namespaces? Based on my very limited understanding Kubernetes is basically a “cloud operating system”. This is why the underlying components like networking and container managers can be swapped out. But I still don’t grok how the subsystems fit together.
ilyt 3y ago
I put stuff in systemd units and route them on HAProxy. For home network it's more than enough. Occasional container if app is designed so badly the authors can't figure out how to make it easy to install or package

mulmen 3y ago

Kubernetes should be called Kafka but the name was already taken. It feels deliberately complicated. I’m honestly curious about Kubernetes and I want to learn more but wow, it really is hard to grok. What does it mean that k3s is “certified”? Should I care? What’s the tradeoff? When you mentioned Traefik my eyes glazed over. Is there a picture of how all this fits together? One not created by a consultant selling their services?

hosh 3y ago
It isn’t deliberately complicated, and once you know why it is designed the way it is, it becomes very capable. The biggest thing to understand about Kubernetes is that you cannot understand it as a normal piece of software. You don’t directly control it, but rather, define the conditions under which a collection of automations respond to things going on. This is how it self heals. The whole system resembles more of a garden, where you are providing the conditions in which the plant grows, but you are not directly causing the plant to grow. For example, if you understand a pod, then the next step is understanding how to keep a pod running, how to detect failures, what go do if it does fail, and how to change out one set of pods for the next. This would be a deployment. You build your understanding by understanding more independent automations. For example, after understanding a deployment, you can look at how horizontal pod autoscalers work. An HPA knows nothing about a deployment and its internal state, yet will inform the deployment how many pods it should be running based on the metrics it sets. The HPA and the deployment scheduler otherwise work independently, and you can indeed write something that functions as an alternate HPA, and Kubernetes will do it. There’s also an addon for cluster autoscaling. The cluster autoscaling knows nothing of deployments or HPA. It only checks pods that are being requested, but cannot be scheduled due to constraints. It then add nodes so that pods can be scheduled. These three different processes operate independently, and don’t share state. They each watch for a limited set of signals and take limited set of actions. If any one of these processes goes down, the system can still limp along. Together, they create a much more complex behavior out of much simpler behaviors.
- mulmen 3y ago
  > It isn’t deliberately complicated, and once you know why it is designed the way it is, it becomes very capable. I’m sure it isn’t and I changed the emphasis to make that line more about the slope of the learning curve. Thanks for this explanation. The inability to grok the internals is really hanging me up. I’ll embrace the simpler “distributions” as a way to understand the higher level constructs. Does it also make sense to ignore the fact that this is built on Linux’s containers and namespaces? Based on my very limited understanding Kubernetes is basically a “cloud operating system”. This is why the underlying components like networking and container managers can be swapped out. But I still don’t grok how the subsystems fit together.

ilyt 3y ago

I put stuff in systemd units and route them on HAProxy. For home network it's more than enough. Occasional container if app is designed so badly the authors can't figure out how to make it easy to install or package