3 ms·

Kubernetes should be called Kafka but the name was already taken. It feels deliberately complicated. I’m honestly curious about Kubernetes and I want to learn

by mulmen 3y ago

Kubernetes should be called Kafka but the name was already taken. It feels deliberately complicated. I’m honestly curious about Kubernetes and I want to learn more but wow, it really is hard to grok. What does it mean that k3s is “certified”? Should I care? What’s the tradeoff? When you mentioned Traefik my eyes glazed over. Is there a picture of how all this fits together? One not created by a consultant selling their services?

hosh 3y ago
It isn’t deliberately complicated, and once you know why it is designed the way it is, it becomes very capable. The biggest thing to understand about Kubernetes is that you cannot understand it as a normal piece of software. You don’t directly control it, but rather, define the conditions under which a collection of automations respond to things going on. This is how it self heals. The whole system resembles more of a garden, where you are providing the conditions in which the plant grows, but you are not directly causing the plant to grow. For example, if you understand a pod, then the next step is understanding how to keep a pod running, how to detect failures, what go do if it does fail, and how to change out one set of pods for the next. This would be a deployment. You build your understanding by understanding more independent automations. For example, after understanding a deployment, you can look at how horizontal pod autoscalers work. An HPA knows nothing about a deployment and its internal state, yet will inform the deployment how many pods it should be running based on the metrics it sets. The HPA and the deployment scheduler otherwise work independently, and you can indeed write something that functions as an alternate HPA, and Kubernetes will do it. There’s also an addon for cluster autoscaling. The cluster autoscaling knows nothing of deployments or HPA. It only checks pods that are being requested, but cannot be scheduled due to constraints. It then add nodes so that pods can be scheduled. These three different processes operate independently, and don’t share state. They each watch for a limited set of signals and take limited set of actions. If any one of these processes goes down, the system can still limp along. Together, they create a much more complex behavior out of much simpler behaviors.
- mulmen 3y ago
  > It isn’t deliberately complicated, and once you know why it is designed the way it is, it becomes very capable. I’m sure it isn’t and I changed the emphasis to make that line more about the slope of the learning curve. Thanks for this explanation. The inability to grok the internals is really hanging me up. I’ll embrace the simpler “distributions” as a way to understand the higher level constructs. Does it also make sense to ignore the fact that this is built on Linux’s containers and namespaces? Based on my very limited understanding Kubernetes is basically a “cloud operating system”. This is why the underlying components like networking and container managers can be swapped out. But I still don’t grok how the subsystems fit together.
  - hosh 3y ago
    The pod is the basic abstraction. The underlying container technology can be swapped out. People have made Kubernetes schedulers for Microsoft technologies and WASM runtimes as well. If you wanted to use microvms, you can. Most people stick with Linux containers. The Linux container/namespace is useful in the sense that operational characteristics matter, and many troubleshooting can end up going deeper and deeper in the stack. But as far as understanding how to _design_ things, knowing how the pure abstractions are composed together lets you be able to apply it to many systems, whether it is Kubernetes or something else. I suggest reading through the first few chapters of Roy Fielding’s PhD dissertation. He is known better for REST, but his dissertation was actually about architectural elements, properties, and constraints, and then applying to hypermedia as an engine of state. Once you know how to reason through computing architecture, you can pick apart any architecture, framework, or paradigm, and know how to work with it, and when you fight against it. You don’t grok a system by finding out how specific parts fit with each other, but rather, how things are consequences of the architectural elements, properties, and constraints. Then, the specific way things work (and don’t work!) together pops out. And don’t be afraid to dive into the messiness behind an abstraction. Knowing and being willing to do that can set you apart in your career.
KronisLV 3y ago
> What does it mean that k3s is “certified”? Should I care? What’s the tradeoff? I think that maybe I should have linked directly to the K3s GitHub, which goes into detail about what it is, what the components are, as well as links to more information about testing: https://github.com/k3s-io/k3s/ https://github.com/k3s-io/k3s/ Kubernetes is a large and complex system, but doing end to end tests aims to ensure that various implementations work correctly, they have pretty in depth docs on all of this, including conformance tests in particular: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md https://github.com/kubernetes/community/blob/master/contribu... So, if a distribution passes all of these, then it's likely that it will give you fewer unpleasant surprises. Here's a list: https://www.cncf.io/certification/software-conformance/ https://www.cncf.io/certification/software-conformance/ > When you mentioned Traefik my eyes glazed over. Well, you typically do need an ingress controller that will allow exposing services to the outside and direct network traffic. Traefik is liked by some, less so by others, but generally works okay - except that its documentation (for K3s in particular) isn't as abundant as it would be when using Nginx. For example, if you want to use a custom SSL/TLS certificate (without ACME) for all domains by default, figuring that out might take a while. That said, if you don't like it, technically you could swap it for something else, however I personally had a cluster hang once while doing that due to some cleanup action getting stuck. There are plenty of options: https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/ https://kubernetes.io/docs/concepts/services-networking/ingr... > Is there a picture of how all this fits together? One not created by a consultant selling their services? I actually found their diagrams to be surprisingly sane: https://docs.k3s.io/architecture https://docs.k3s.io/architecture However, do be warned that there are many different ways how to build clusters that might have significant differences. K3s is just one of the Kubernetes distros - that attempt to provide an opinionated view on how to mix and match the components out there to end up with something that will let you use Kubernetes and have things actually work as expected at the end of the day. A bit like how we have OS distros like Debian, Ubuntu, RHEL and so on.

by mulmen 3y ago

hosh 3y ago

It isn’t deliberately complicated, and once you know why it is designed the way it is, it becomes very capable. The biggest thing to understand about Kubernetes is that you cannot understand it as a normal piece of software. You don’t directly control it, but rather, define the conditions under which a collection of automations respond to things going on. This is how it self heals. The whole system resembles more of a garden, where you are providing the conditions in which the plant grows, but you are not directly causing the plant to grow. For example, if you understand a pod, then the next step is understanding how to keep a pod running, how to detect failures, what go do if it does fail, and how to change out one set of pods for the next. This would be a deployment. You build your understanding by understanding more independent automations. For example, after understanding a deployment, you can look at how horizontal pod autoscalers work. An HPA knows nothing about a deployment and its internal state, yet will inform the deployment how many pods it should be running based on the metrics it sets. The HPA and the deployment scheduler otherwise work independently, and you can indeed write something that functions as an alternate HPA, and Kubernetes will do it. There’s also an addon for cluster autoscaling. The cluster autoscaling knows nothing of deployments or HPA. It only checks pods that are being requested, but cannot be scheduled due to constraints. It then add nodes so that pods can be scheduled. These three different processes operate independently, and don’t share state. They each watch for a limited set of signals and take limited set of actions. If any one of these processes goes down, the system can still limp along. Together, they create a much more complex behavior out of much simpler behaviors.

mulmen 3y ago
> It isn’t deliberately complicated, and once you know why it is designed the way it is, it becomes very capable. I’m sure it isn’t and I changed the emphasis to make that line more about the slope of the learning curve. Thanks for this explanation. The inability to grok the internals is really hanging me up. I’ll embrace the simpler “distributions” as a way to understand the higher level constructs. Does it also make sense to ignore the fact that this is built on Linux’s containers and namespaces? Based on my very limited understanding Kubernetes is basically a “cloud operating system”. This is why the underlying components like networking and container managers can be swapped out. But I still don’t grok how the subsystems fit together.
- hosh 3y ago
  The pod is the basic abstraction. The underlying container technology can be swapped out. People have made Kubernetes schedulers for Microsoft technologies and WASM runtimes as well. If you wanted to use microvms, you can. Most people stick with Linux containers. The Linux container/namespace is useful in the sense that operational characteristics matter, and many troubleshooting can end up going deeper and deeper in the stack. But as far as understanding how to _design_ things, knowing how the pure abstractions are composed together lets you be able to apply it to many systems, whether it is Kubernetes or something else. I suggest reading through the first few chapters of Roy Fielding’s PhD dissertation. He is known better for REST, but his dissertation was actually about architectural elements, properties, and constraints, and then applying to hypermedia as an engine of state. Once you know how to reason through computing architecture, you can pick apart any architecture, framework, or paradigm, and know how to work with it, and when you fight against it. You don’t grok a system by finding out how specific parts fit with each other, but rather, how things are consequences of the architectural elements, properties, and constraints. Then, the specific way things work (and don’t work!) together pops out. And don’t be afraid to dive into the messiness behind an abstraction. Knowing and being willing to do that can set you apart in your career.

KronisLV 3y ago

> What does it mean that k3s is “certified”? Should I care? What’s the tradeoff? I think that maybe I should have linked directly to the K3s GitHub, which goes into detail about what it is, what the components are, as well as links to more information about testing: https://github.com/k3s-io/k3s/ https://github.com/k3s-io/k3s/ Kubernetes is a large and complex system, but doing end to end tests aims to ensure that various implementations work correctly, they have pretty in depth docs on all of this, including conformance tests in particular: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md https://github.com/kubernetes/community/blob/master/contribu... So, if a distribution passes all of these, then it's likely that it will give you fewer unpleasant surprises. Here's a list: https://www.cncf.io/certification/software-conformance/ https://www.cncf.io/certification/software-conformance/ > When you mentioned Traefik my eyes glazed over. Well, you typically do need an ingress controller that will allow exposing services to the outside and direct network traffic. Traefik is liked by some, less so by others, but generally works okay - except that its documentation (for K3s in particular) isn't as abundant as it would be when using Nginx. For example, if you want to use a custom SSL/TLS certificate (without ACME) for all domains by default, figuring that out might take a while. That said, if you don't like it, technically you could swap it for something else, however I personally had a cluster hang once while doing that due to some cleanup action getting stuck. There are plenty of options: https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/ https://kubernetes.io/docs/concepts/services-networking/ingr... > Is there a picture of how all this fits together? One not created by a consultant selling their services? I actually found their diagrams to be surprisingly sane: https://docs.k3s.io/architecture https://docs.k3s.io/architecture However, do be warned that there are many different ways how to build clusters that might have significant differences. K3s is just one of the Kubernetes distros - that attempt to provide an opinionated view on how to mix and match the components out there to end up with something that will let you use Kubernetes and have things actually work as expected at the end of the day. A bit like how we have OS distros like Debian, Ubuntu, RHEL and so on.