33 ms·
The Missing Kubernetes Type System
- zeckalpha 5y agoFully agree! Here’s my post on this problem space from nearly a decade ago: https://kyle.marek-spartz.org/posts/2014-05-08-your-service-as-a-functor.html https://kyle.marek-spartz.org/posts/2014-05-08-your-service-...
- mleonhard 5y agoThat's a nice formalization of services. Did you try implementing it?
- jzelinskie 5y agoI've been a PM for Kubernetes at two of the major contributing companies and helped popularize Operators. The project that will replace Kubernetes will be something that provides builtins that compose together to EXTEND the system. With such a foundation, you'll be given the flexibility to evolve over time and correct design flaws. Kubernetes was in a rush to beat competitors out in the early orchestration wars, so they focused on primitives that people needed to deploy "today". I think they did a good job, but the focus wasn't on extensibility. In an ideal world (and I've proposed), there be a long term SIG/WG that experimented with custom controllers to add new primitives that focused on EXTENDING Kubernetes. Most companies at the time wanted to tighten up the core APIs, not realizing that they should be skating towards where the puck is going. Now every major Kubernetes provider is past the point of just getting clusters into folks' hands and are realizing they need to add more functionality, but have no means that doesn't come at the cost of explosive complexity.
- the_duke 5y agoOut of curiosity, and particularly because I'm working on a Webassembly based alternative to Kubernetes: What would you do differently if you could start from scratch?
- readams 5y agoVery curious where you see the advantage of using Webassembly there. Presumably this is not running in a browser.
- the_duke 5y agoWebassembly is also gaining steam for server workloads due to several advantages (less overhead, better capability based security, composability, ...). See Spin [1], Wasmcloud [2], Lunatic [3], etc. My system is based on a distributed Webassembly runtime. The reason for taking inspiration from Kubernetes is making deployment of distributed workloads on that runtime easy. A nice benefit for a Kubernetes alike system is that the equivalent to operators and controllers can be much more light-weight WASM actors that are easier to develop, deploy and scale. [1] https://github.com/fermyon/spin https://github.com/fermyon/spin [2] https://github.com/wasmCloud/wasmCloud https://github.com/wasmCloud/wasmCloud [3] https://github.com/lunatic-solutions/lunatic https://github.com/lunatic-solutions/lunatic
- kitd 5y agoEven better (and I expect you're aware of this already), you can run Webassembly workloads under Kubernetes control: https://krustlet.dev/ https://krustlet.dev/
- the_duke 5y agoYeah, krustlet is great for integrating into existing Kube deployments. I'm just tackling the problem from the other side.
- andrewallbright 5y agoWebassembly as a desirable server side technology... I suppose I need to recontextualize what I understand webassembly to be. Hmm.
- traverseda 5y ago
- mountainriver 5y agoThis is exactly right. I don’t think it’ll happen within k8s because if you’ve ever worked with sig-api you’ll know they aren’t big on innovation. The biggest problem with kube is the declarative interfaces aren’t enough for most problems. You need imperative methods as well and the only way to sort of do this is with aggregated API servers (yuck!) Even the core Pod resource even has a /logs endpoint which is imperative. Also kubectl is a mess, verb->noun makes no sense at all.
- EdwardDiego 5y agoI'm paid to work on a FOSS operator, and I couldn't quite discern a) the issue and b) the proposed solution. Is the author wanting to move away from K8s' Go style object orientation (structs and functions that act on them) to more of a Java style OO where objects express both data and behaviour?
- MrBuddyCasino 5y agoCouldn’t tell either. I’m familiar with the conceptual building blocks, but I‘d have a hard time extracting a tl;dr.
- morelisp 5y agoI'm not sure the analogy with types is at all helpful in explaining this. If I understand the author right, the critique is that Kubernetes wants to present a unified, declarative model for arbitrary orchestration behavior. To do this, it uses controllers/operators/CRDs/etc, but the code within these controllers is often some of the imperativiest-imperativing stuff you've ever seen. This is "fine" for some core primitives, but as you demand more complex orchestrations out of k8s it's more likely you have to write these yourself, and it's messy and "hard", beyond what most teams can or want to invest, because of that. (Up to this point, I agree fully.) But then the "types" analogy comes to the fore, and we end up with a system that's even more complex, a kind of HKT for declarative behaviors. I'm not sure it really solves the implementation problems to begin with, but even if it does - I predict the teams struggling to implement custom operators are going to slam into this new wall even harder trying to model their operational processes like this. How long until we finally admit a lot of things an operator (human) does are imperative, and therefore the k8s "declarative only" model is fundamentally broken? What we need is more carefully-chosen imperative escape hatches, not more API options.
- moondev 5y agoI appreciate your summary. I wonder if something like metacontroller aligns with this problem space. https://metacontroller.github.io/metacontroller/intro.html https://metacontroller.github.io/metacontroller/intro.html
- bogomipz 5y ago>"But then the "types" analogy comes to the fore, and we end up with a system that's even more complex, a kind of HKT for declarative behaviors." I agree with all of your points. What is HKT here though?
- c0balt 5y agoNot parent but most likely Higher-kinded Type[0] [0]: https://en.m.wikipedia.org/wiki/Kind_(type_theory) https://en.m.wikipedia.org/wiki/Kind_(type_theory)
- mountainriver 5y ago