4 ms·

So, as as a serious answer, yes. The problem isn't deployments, it's that k8s doesn't only do deployments. It does a ton of stuff, and deployments are a small

by C4stor 6y ago

So, as as a serious answer, yes. The problem isn't deployments, it's that k8s doesn't only do deployments. It does a ton of stuff, and deployments are a small part of it. If you only wanted to use k8s for deploying your code, well, you can't. That's what's was great with the UNIX programs philosophy, and what is lost in kubernetes. You can't pick the feature you need, and the feature you want may well break because of a problem in another part of the system you don't even know existed (hello persistent volumes flagged for deletions, resource allocation failing, ...) No single Kubernetes feature may be overcomplicated, but k8s as a whole is for sure extremely complex.

jrockway 6y ago
What is the unnecessary part, in your opinion? I think the core is very good. You need a pool of machines, you need to be able to schedule workloads, you need the workloads to be able to talk to each other over the network, you need the cluster to reach a consensus about what state it's in, etc. That is complicated, but basically mandatory. Intrinsically, I think you take on a lot of problems by increasing the number of computers you control. Once you have decided that you need more than 0 computers, you are in for a world of hurt -- that's the entire field of software engineering. Once you have decided to make the jump from 1 computer to 2 computers, you now have to contend with the entire field of distributed systems, just so that you can serve your website when one machine fails, or you want to upgrade Linux, or whatever. At some point, even "small" companies have 10 engineers and 10 computers, and that is right about when "some guy will handle this for me" fails and you start needing software to manage things like scheduling or deployments. You can certainly make it work for longer than that without using Kubernetes, but the returns start diminishing very quickly. There are absolutely quirks of Kubernetes that have caused users a lot of unnecessary hurt -- mandating a 5 node etcd cluster when they could have bought an RDS instance for $70/month or something. That seems to be being fixed, though. The way I see it is that people run into trouble with the parts that are underspecified. Ingress is my classic example -- it only provides API surface for the simplest of HTTP routing operations, and it's too simple for any real-world application. The result is that people hack in their own ingress controllers in horrifying ways ("just put an nginx.conf snippet in an annotation!") and that bites a lot of users. It goes further than things like Ingress; who is providing monitoring, tracing, and authentication? Bring your own! Who gives you a container registry? Bring your own. How do you code review changes to your cluster? Bring your own gitops! (This is all in addition to the things I mentioned in my original comment.) There are too many things that Kubernetes should probably do because a lot of people need them, but those people have to cobble together a bunch of unrelated components to get what they need. That is nice! But one way should win, and it would save people a lot of time.
- C4stor 6y ago
  > You need a pool of machines, you need to be able to schedule workloads, you need the workloads to be able to talk to each other over the network, you need the cluster to reach a consensus about what state it's in Well, no ? I need to be able to start an app, and when that app says 'I'm ready to run', shutdown the old version. So a the extreme version, I'd just need to ensure all my apps have 2 open endpoint for "readiness" and "kill", and one way to register themselves in some kind of central db of "what's running". That's it, that's an automated deployment solution. Of course, it doesn't really do a lot, but done well, it would be something already ! I don't need a pool of machines, or at least that's not the matter for a "deployment solution". I don't need scheduling, and I certainly don't want something to interfere with how I handle network in my app. I don't say that all of this isn't good or often needed, but it's not atomic to the need of "deploying an app". And then the problem arises that the feature set of k8s that is neither complete wrt some utopic "ultimate end all solution" (because realistically, it can't be), nor atomic at all on what it does. And solutions like k8s end up somewhere in the middle, never finished, always updating, and well, complicated !

by C4stor 6y ago

jrockway 6y ago

What is the unnecessary part, in your opinion? I think the core is very good. You need a pool of machines, you need to be able to schedule workloads, you need the workloads to be able to talk to each other over the network, you need the cluster to reach a consensus about what state it's in, etc. That is complicated, but basically mandatory. Intrinsically, I think you take on a lot of problems by increasing the number of computers you control. Once you have decided that you need more than 0 computers, you are in for a world of hurt -- that's the entire field of software engineering. Once you have decided to make the jump from 1 computer to 2 computers, you now have to contend with the entire field of distributed systems, just so that you can serve your website when one machine fails, or you want to upgrade Linux, or whatever. At some point, even "small" companies have 10 engineers and 10 computers, and that is right about when "some guy will handle this for me" fails and you start needing software to manage things like scheduling or deployments. You can certainly make it work for longer than that without using Kubernetes, but the returns start diminishing very quickly. There are absolutely quirks of Kubernetes that have caused users a lot of unnecessary hurt -- mandating a 5 node etcd cluster when they could have bought an RDS instance for $70/month or something. That seems to be being fixed, though. The way I see it is that people run into trouble with the parts that are underspecified. Ingress is my classic example -- it only provides API surface for the simplest of HTTP routing operations, and it's too simple for any real-world application. The result is that people hack in their own ingress controllers in horrifying ways ("just put an nginx.conf snippet in an annotation!") and that bites a lot of users. It goes further than things like Ingress; who is providing monitoring, tracing, and authentication? Bring your own! Who gives you a container registry? Bring your own. How do you code review changes to your cluster? Bring your own gitops! (This is all in addition to the things I mentioned in my original comment.) There are too many things that Kubernetes should probably do because a lot of people need them, but those people have to cobble together a bunch of unrelated components to get what they need. That is nice! But one way should win, and it would save people a lot of time.

C4stor 6y ago
> You need a pool of machines, you need to be able to schedule workloads, you need the workloads to be able to talk to each other over the network, you need the cluster to reach a consensus about what state it's in Well, no ? I need to be able to start an app, and when that app says 'I'm ready to run', shutdown the old version. So a the extreme version, I'd just need to ensure all my apps have 2 open endpoint for "readiness" and "kill", and one way to register themselves in some kind of central db of "what's running". That's it, that's an automated deployment solution. Of course, it doesn't really do a lot, but done well, it would be something already ! I don't need a pool of machines, or at least that's not the matter for a "deployment solution". I don't need scheduling, and I certainly don't want something to interfere with how I handle network in my app. I don't say that all of this isn't good or often needed, but it's not atomic to the need of "deploying an app". And then the problem arises that the feature set of k8s that is neither complete wrt some utopic "ultimate end all solution" (because realistically, it can't be), nor atomic at all on what it does. And solutions like k8s end up somewhere in the middle, never finished, always updating, and well, complicated !