9 ms·

“etcd, apiserver, and controllers.” …and containerd and csi plugins and kubelet and cni plugins and kubectl and kube-proxy and ingresses and load balancers…

igmor 2y ago
These components are very different in complexity and scope. Let's be real: a seasoned developer is mostly familiar with load balancers and ingress controllers, so this will be mostly about naming and context. I agree though once you learn about k8s it becomes less mysterious but that also means the author hasn't pushed it to the limits. Outages in the control plane could be pretty nasty and it is easy to have them by creating an illusion everything is kind of free in k8s.
- nicoburns 2y ago
  A really simple setup for many smaller organisations wouldn't have a load balancer at all.
  - lakomen 2y ago
    [dead]
  - darkwater 2y ago
    No load balancer means... entering one node only? Doing DNS RR over all the nodes? If you don't have a load balancer in front, why are you even using Kubernetes? Deploy a single VM and call it a day! I mean, in my homelab I do have Kubernetes and no LB in front, but it's a homelab for fun and learn K8s internals. But in a professional environment...
  - dilyevsky 2y ago
    No code at all even - just use excel
zeroq 2y ago
typical how to program an owl: step one: draw a circle step two: import the rest of the owl
remram 2y ago
And system calls and filesystems and sockets and LVM and... Sure at some point there are too many layers to count but I wouldn't say any of this is "Kubernetes". What people tend to be hung about is the difficulty of Kubernetes compared to `docker run` or `docker compose up`. That is what I am surprised about. I never had any issue with kubelet, or kube-proxy, or CSI plugins, or CNI plugins. That is after years of running a multi-tenant cluster in a research institution. I think about those about as much as I think about ext4, runc, or GRUB.
- ffsm8 2y ago
  But you just said that you had issues with ceph? How is that not a CSI problem? And CNI problems are extremely normal. Pretty much anyone that didn't just use weavenet and called it a day has had to spend quiet a bit of time to figure it out. If you already know networking by heart it's obviously going to be easier, but few devs do.
  - freedomben 2y ago
    Very fair, although with managed services which are increasingly available, you don't typically need to think about CSI or CNI.
    - deleted 2y ago
      [deleted]
    - cuu508 2y ago
      Hence > Kubernetes is not the first thing that comes to mind when I think of "understanding where their code is running and what it's doing"...
      - remram 2y ago
        CSI and CNI do about as much magic as `docker volume` and `docker network`. People act like their web framework and SQL connection pooler and stuff are so simple, while Kubernetes is complex and totally inscrutable for mortals, and I don't get it. It has a couple of moving parts, but it is probably simpler overall than SystemD.
motorest 2y ago
> …and containerd and csi plugins and kubelet and cni plugins (...) Do you understand you're referring to optional components and add-ons? > and kubectl You mean the command line interface that you optionally use if you choose to do so? > and kube-proxy and ingresses and load balancers… Do you understand you're referring to whole classes of applications you run on top of Kubernetes? I get it that you're trying to make a mountain out of a mole hill. Just understand that you can't argue that something is complex by giving as your best examples a bunch of things that aren't really tied to it. It's like trying to claim Windows is hard, and then your best example is showing a screenshot of AutoCAD.
- allarm 2y ago
  How’s kubelet and cni are “optional components”? What do you mean by that?
  - p_l 2y ago
    kubelet isn't, but CNI technically is (or can be abstracted to minimum, I think old network support might have been removed from kubelet nowadays)
  - remram 2y ago
    CNI is optional, you can have workloads bind ports on the host rather than use an overlay network (though CNI plugins and kube-proxy are extremely simple and reliable in my experience, they use VXLAN and iptables which are built into the kernel and that you already use in any organization who might run a cluster, or the basic building blocks of your cloud provider). CSI is optional, you can just not use persistent storage (use the S3 API or whatever) or declare persistentvolumes that are bound to a single or group of machines (shared NFS mount or whatever). I don't know how GP thinks you could run without the other bits though. You do need kubelet and a container runtime.
donutshop 2y ago
... and kubernetes networking, service mesh, secrets management
- chronid 2y ago
  You arent' forced to use service mesh and complex secrets management schemes. If you add them to the cluster is because you value what they offer you. It's the same thing as kubernetes itself - I'm not sure what people are complaining about, if you don't need what kubernetes offers, just don't use it. Go back to good ol' corsync/pacemaker clusters with XML and custom scripts to migrate IPs and set up firewall rules (and if you have someone writing them for you, why don't you have people managing your k8s clusters?). Or buy something from a cloud provider that "just works" and eventually go down in flames with their indian call centers doing their best but with limited access to engineering to understand why service X is misbehaving for you and trashing your customer's data. It's trade-offs all the way.

igmor 2y ago

These components are very different in complexity and scope. Let's be real: a seasoned developer is mostly familiar with load balancers and ingress controllers, so this will be mostly about naming and context. I agree though once you learn about k8s it becomes less mysterious but that also means the author hasn't pushed it to the limits. Outages in the control plane could be pretty nasty and it is easy to have them by creating an illusion everything is kind of free in k8s.

nicoburns 2y ago
A really simple setup for many smaller organisations wouldn't have a load balancer at all.
- lakomen 2y ago
  [dead]
- darkwater 2y ago
  No load balancer means... entering one node only? Doing DNS RR over all the nodes? If you don't have a load balancer in front, why are you even using Kubernetes? Deploy a single VM and call it a day! I mean, in my homelab I do have Kubernetes and no LB in front, but it's a homelab for fun and learn K8s internals. But in a professional environment...
- dilyevsky 2y ago
  No code at all even - just use excel

zeroq 2y ago

typical how to program an owl: step one: draw a circle step two: import the rest of the owl

remram 2y ago

And system calls and filesystems and sockets and LVM and... Sure at some point there are too many layers to count but I wouldn't say any of this is "Kubernetes". What people tend to be hung about is the difficulty of Kubernetes compared to `docker run` or `docker compose up`. That is what I am surprised about. I never had any issue with kubelet, or kube-proxy, or CSI plugins, or CNI plugins. That is after years of running a multi-tenant cluster in a research institution. I think about those about as much as I think about ext4, runc, or GRUB.

ffsm8 2y ago
But you just said that you had issues with ceph? How is that not a CSI problem? And CNI problems are extremely normal. Pretty much anyone that didn't just use weavenet and called it a day has had to spend quiet a bit of time to figure it out. If you already know networking by heart it's obviously going to be easier, but few devs do.
- freedomben 2y ago
  Very fair, although with managed services which are increasingly available, you don't typically need to think about CSI or CNI.
  - deleted 2y ago
    [deleted]
  - cuu508 2y ago
    Hence > Kubernetes is not the first thing that comes to mind when I think of "understanding where their code is running and what it's doing"...
    - remram 2y ago
      CSI and CNI do about as much magic as `docker volume` and `docker network`. People act like their web framework and SQL connection pooler and stuff are so simple, while Kubernetes is complex and totally inscrutable for mortals, and I don't get it. It has a couple of moving parts, but it is probably simpler overall than SystemD.

motorest 2y ago

> …and containerd and csi plugins and kubelet and cni plugins (...) Do you understand you're referring to optional components and add-ons? > and kubectl You mean the command line interface that you optionally use if you choose to do so? > and kube-proxy and ingresses and load balancers… Do you understand you're referring to whole classes of applications you run on top of Kubernetes? I get it that you're trying to make a mountain out of a mole hill. Just understand that you can't argue that something is complex by giving as your best examples a bunch of things that aren't really tied to it. It's like trying to claim Windows is hard, and then your best example is showing a screenshot of AutoCAD.

allarm 2y ago
How’s kubelet and cni are “optional components”? What do you mean by that?
- p_l 2y ago
  kubelet isn't, but CNI technically is (or can be abstracted to minimum, I think old network support might have been removed from kubelet nowadays)
- remram 2y ago
  CNI is optional, you can have workloads bind ports on the host rather than use an overlay network (though CNI plugins and kube-proxy are extremely simple and reliable in my experience, they use VXLAN and iptables which are built into the kernel and that you already use in any organization who might run a cluster, or the basic building blocks of your cloud provider). CSI is optional, you can just not use persistent storage (use the S3 API or whatever) or declare persistentvolumes that are bound to a single or group of machines (shared NFS mount or whatever). I don't know how GP thinks you could run without the other bits though. You do need kubelet and a container runtime.

donutshop 2y ago

... and kubernetes networking, service mesh, secrets management

chronid 2y ago
You arent' forced to use service mesh and complex secrets management schemes. If you add them to the cluster is because you value what they offer you. It's the same thing as kubernetes itself - I'm not sure what people are complaining about, if you don't need what kubernetes offers, just don't use it. Go back to good ol' corsync/pacemaker clusters with XML and custom scripts to migrate IPs and set up firewall rules (and if you have someone writing them for you, why don't you have people managing your k8s clusters?). Or buy something from a cloud provider that "just works" and eventually go down in flames with their indian call centers doing their best but with limited access to engineering to understand why service X is misbehaving for you and trashing your customer's data. It's trade-offs all the way.