2 ms·

This is a pretty ok framing. The author's already had to reply to some backblow against the idea but generally the big picture rings true enough, isnt horrifica

This is a pretty ok framing. The author's already had to reply to some backblow against the idea but generally the big picture rings true enough, isnt horrifically wrong. But while true & a reasonable framing, this view doesnt cover the most important thing about the kubernetes "operating system". Yes unlike many other cloud systems Kubernetes has pretty wide coverage of concepts it has under it's roof. Versus serving a purpose or two well, it, like an OS is a fairly general & comprehensive platform, with the capabilities you'd need for most tasks, & built in OS extensibility via CRDs. This is all trye enough but still shy of how we ought think about Kubernetes versus other things, what came before, cloud or no cloud, os or other. Kubernetes is autonomic. Kubernetes, through a variety of controllers/operators, enacts all changes to the system. Rather than being, as most automation or other aoftware, a tool wielded by the user to make ends happen Kubernetes wields itself, & the user is simply stating what they want, makes their intent known, & Kubernetes makes sure that happens & stays that way. Taking people out of the loop of having to learn how to do computers, how to build a house themselves- Kubernetes knows how to build what you tell it. It's something so unlike software of any sort which came before, this ongoing (as opposed to single shot), autonomic (self maintaining), self-operating system, and I cannot contrast strongly enough how liberating it is to have the machines working for you, adjusting themselves, based off the manifest of inventory you've told it you want. It's a totally different operational paradigm, beyond just a system & having developed a weak internal agency (a control loop & it's suite of tools to stay on happy paths through the loop), and it means way way less figuring out how to run the dozens of various whirligigs and gizmos that comprise the computing environment. Kubernetes is more than an operating system; it is a self-governing control loop.

chousuke 5y ago
Kubernetes, by itself, does not really know how to "do" anything, but it provides a framework for writing components that do (controllers) in response to new resources being declared (I want to say "created" but nothing "real" exists until a controller does something) and updated.
- nyellin 5y ago
  You can say created! It exists in etcd just hasn't been materialized in other ways yet.
  - chousuke 5y ago
    Yeah, I wanted to emphasize the difference between the kubernetes resource and the actual things that the controllers create based on the existence and attributes of those resources. I suppose "materialization" is a good word for it; you create a resource, and if it's something that actually needs something to execute, it is eventually materialized as a container scheduled to run on a kubelet, or an EBS volume, or some other actual thing that the appropriate controller creates. I like the core idea of Kubernetes (eventually consistent declarative infrastructure) though I'm not fully sold on the implementation and the tooling around it.
    - nyellin 5y ago
      Yeah fair point. Regarding the implementation and tooling, what's lacking in your opinion?
      - chousuke 5y ago
        Well, some thoughts in no particular order: 0. YAML everywhere. Augh. 1. Vanilla kubernetes by itself doesn't really do much; everyone basically assumes you'll use a cluster in a cloud somewhere that comes with all kinds of add-ons and functionality on top of the default. If you want to run an on-premises cluster, once you start adding external components to a Kubernetes cluster to make it actually useful, it starts getting complicated. 2. I don't think configuration management of the resources you do put in a cluster is a fully solved problem, especially if you depend on CRDs. 3. kubectl being context-sensitive is rather annoying. You basically can't share a kubeconfig between two terminal sessions without severe danger of footgunning. 4. If you just use Helm packages with defaults, you'll end up running software with very collision-prone UIDs and security contexts, so two separate components that you deploy can end up running as the same user on a host. I don't think that's good security design, but the system doesn't encourage you to do the correct thing. 5. All the "easy" tooling is basically just "run this magic incantation and it will do things for you". The culture around how things are done seem to encourage ignorance about what you're actually deploying. Many operations also modify cluster state and thus should be managed and not done willy-nilly, but that's also ignored by default. See also #2 Kubernetes has its upsides (something like ArgoCD is very nice to use when set up properly) but I guess my main beef with it is that it can make things look easy that actually aren't, and mislead its users.
mooreds 5y ago
Maybe I miss your point, but how is the autonomic nature of kubernetes different than, say MacOS? I don't know how to write a file or install a program, I just tell MacOS to do this and it happens magically. Is it the level of abstraction that is different? Or are you saying that kubernetes lets you do this at a different scale or with different kinds of applications than you could before?
- rektide 5y ago
  Much of the history of computing is edge-triggered signalling. You instruct the computer to copy a file by signalling a mousedown in one place, dragging your file (a stream of movement signals), and mouse-uping. These events are processed online and you have your output then you are done. The computer keeps no memory of what you or it did, the system is just in a new state, by you having performed this operation with the os tools. Kubernetes is level-triggered. If you could say that you want a certain file in a certain spot, the computer might be able to have choices for how to make it so. Maybe it looks in other directories. Maybe it downloads it via http or bittorrent. Maybe it scans your archives. If the hard drive dies, it can reget the file. If your sibling deletes the file, the computer can rebuild it. There's snapshotting policies it can setup on your behalf to help. There's records about how & where & when it pulled the file in from. This is all an elaboration on autonomic processes, on Kubernetes maintaining state for you, letting it do the operations to give you what you asked for. It detects when the level is not right/monitors itself, & detects when it needs to trigger (level triggered) control. it has it's own toolkits to do the work for you, and you are merely registering your ask. yes you might not fully understand the tool you are using & the full depth of a file system, but even as an end user you are doing the work in tbe conventional macOS model. abstracting the work away from the user, & using a general repeatable practice for declaring your intent (manifests) frees you from having to learn a panolopy of tools & systems, each unique an different (confinung ourselves just to this one example, a file manager, a web browser, a bittorrent client, a usenet client, a snapshot tool, file search system). and it maintains itself, keeps working, consistently & with explanation, where-as most computers defy explanation: what scripts have been run, what the history is is unknown. by making a compact - letting the machine do the the work instead of us doing it- we have central bookkeeping of what our system really is, radically reshaping the clarity of what we have & what's going on. these skills being portable across problem domains is a third ultra-strong benefit. almost all aspects of the world get maintained via manifests. learning something new looks similar to & used many of the same tools as what you did to setup the last tbing. this is not really an autonomic benefit, byt is a huge part of the win. generalized autonomic system is a huge improvement versus shepparding our individual systems carefully forward.

chousuke 5y ago

Kubernetes, by itself, does not really know how to "do" anything, but it provides a framework for writing components that do (controllers) in response to new resources being declared (I want to say "created" but nothing "real" exists until a controller does something) and updated.

nyellin 5y ago
You can say created! It exists in etcd just hasn't been materialized in other ways yet.
- chousuke 5y ago
  Yeah, I wanted to emphasize the difference between the kubernetes resource and the actual things that the controllers create based on the existence and attributes of those resources. I suppose "materialization" is a good word for it; you create a resource, and if it's something that actually needs something to execute, it is eventually materialized as a container scheduled to run on a kubelet, or an EBS volume, or some other actual thing that the appropriate controller creates. I like the core idea of Kubernetes (eventually consistent declarative infrastructure) though I'm not fully sold on the implementation and the tooling around it.
  - nyellin 5y ago
    Yeah fair point. Regarding the implementation and tooling, what's lacking in your opinion?
    - chousuke 5y ago
      Well, some thoughts in no particular order: 0. YAML everywhere. Augh. 1. Vanilla kubernetes by itself doesn't really do much; everyone basically assumes you'll use a cluster in a cloud somewhere that comes with all kinds of add-ons and functionality on top of the default. If you want to run an on-premises cluster, once you start adding external components to a Kubernetes cluster to make it actually useful, it starts getting complicated. 2. I don't think configuration management of the resources you do put in a cluster is a fully solved problem, especially if you depend on CRDs. 3. kubectl being context-sensitive is rather annoying. You basically can't share a kubeconfig between two terminal sessions without severe danger of footgunning. 4. If you just use Helm packages with defaults, you'll end up running software with very collision-prone UIDs and security contexts, so two separate components that you deploy can end up running as the same user on a host. I don't think that's good security design, but the system doesn't encourage you to do the correct thing. 5. All the "easy" tooling is basically just "run this magic incantation and it will do things for you". The culture around how things are done seem to encourage ignorance about what you're actually deploying. Many operations also modify cluster state and thus should be managed and not done willy-nilly, but that's also ignored by default. See also #2 Kubernetes has its upsides (something like ArgoCD is very nice to use when set up properly) but I guess my main beef with it is that it can make things look easy that actually aren't, and mislead its users.

mooreds 5y ago

Maybe I miss your point, but how is the autonomic nature of kubernetes different than, say MacOS? I don't know how to write a file or install a program, I just tell MacOS to do this and it happens magically. Is it the level of abstraction that is different? Or are you saying that kubernetes lets you do this at a different scale or with different kinds of applications than you could before?

rektide 5y ago
Much of the history of computing is edge-triggered signalling. You instruct the computer to copy a file by signalling a mousedown in one place, dragging your file (a stream of movement signals), and mouse-uping. These events are processed online and you have your output then you are done. The computer keeps no memory of what you or it did, the system is just in a new state, by you having performed this operation with the os tools. Kubernetes is level-triggered. If you could say that you want a certain file in a certain spot, the computer might be able to have choices for how to make it so. Maybe it looks in other directories. Maybe it downloads it via http or bittorrent. Maybe it scans your archives. If the hard drive dies, it can reget the file. If your sibling deletes the file, the computer can rebuild it. There's snapshotting policies it can setup on your behalf to help. There's records about how & where & when it pulled the file in from. This is all an elaboration on autonomic processes, on Kubernetes maintaining state for you, letting it do the operations to give you what you asked for. It detects when the level is not right/monitors itself, & detects when it needs to trigger (level triggered) control. it has it's own toolkits to do the work for you, and you are merely registering your ask. yes you might not fully understand the tool you are using & the full depth of a file system, but even as an end user you are doing the work in tbe conventional macOS model. abstracting the work away from the user, & using a general repeatable practice for declaring your intent (manifests) frees you from having to learn a panolopy of tools & systems, each unique an different (confinung ourselves just to this one example, a file manager, a web browser, a bittorrent client, a usenet client, a snapshot tool, file search system). and it maintains itself, keeps working, consistently & with explanation, where-as most computers defy explanation: what scripts have been run, what the history is is unknown. by making a compact - letting the machine do the the work instead of us doing it- we have central bookkeeping of what our system really is, radically reshaping the clarity of what we have & what's going on. these skills being portable across problem domains is a third ultra-strong benefit. almost all aspects of the world get maintained via manifests. learning something new looks similar to & used many of the same tools as what you did to setup the last tbing. this is not really an autonomic benefit, byt is a huge part of the win. generalized autonomic system is a huge improvement versus shepparding our individual systems carefully forward.