3 ms·

> And thus I ask again, what's better? I mean, if you're asking me to prescribe a one-size-fits-all infrastructure solution for any problem, that's not a thing

> And thus I ask again, what's better? I mean, if you're asking me to prescribe a one-size-fits-all infrastructure solution for any problem, that's not a thing. The answer is "it depends". What's better is to understand the mechanisms at work and use what you need to get a robust, reliable solution. If I were to prescribe a general infrastructure platform for your generic web app, the high-level would be a) use something for infrastructure-as-code to define the resources needed, so they can be rebuilt/spawned in new environments on demand; b) use something for config management and image construction, ideally something like NixOS that has reproduceability and dependency encapsulation as a fundamental part of the OS; c) use a production-grade load balancer like haproxy or Envoy and configure it properly for the infrastructure that it sits on top of; d) use a production-grade web server to serve requests, which is probably either Apache or nginx. Note that just saying "boom k8s" doesn't really resolve these concerns. k8s schedules, routes, and executes arbitrary containers, which have usually been built by developers who don't know what they're doing, and which are likely to contain tons of random outdated junk as a result of cascading FROMs in the Dockerfile and stray files in the developer and/or CI pipeline's build context (which are frequently sensitive btw). Container images should be just like any other image running in prod: constructed by competent admins and controlled with an appropriate configuration and patch management mechanism. The fact that applying those changes executes an entirely new image instead of just restarting a service is really an implementation detail, it's not a solution to anything. Your chosen k8s ingress is probably using nginx, haproxy, or Envoy under the covers, and you have to tune that either way, whether you call that a livenessProbe or a health check. There's nothing fundamentally better about the k8s YAML for this than the actual configuration file (and indeed, in the early days of k8s, I was hacking through the alpha-stage ingress plugin and editing haproxy configs by hand anyway), though I suppose if you have a use case where half of your applications need one load balancer and the other half need another, you may get some benefit here. That's pretty rare, though. If you ever do hit appreciable load, you may find that your container's `npm start` server has some inadequacies for which "omg pay $CLOUDPROVIDER more money and spin more pods" may not be an adequate solution. And lastly, k8s doesn't enter the picture until you have something to run it on so it doesn't do anything for your infrastructure-as-code problem, even if the answer to that is just "rent a Kube cluster from Google" -- something still has to actually go rent that, and it should be scripted. So I mean, yeah, if you like Kubernetes because Helm charts are convenient, by all means go ahead and run Kubernetes. Just don't pretend like that magically solves the problems involved in a robust architecture, especially if you ever expect to tune or profile your systems.

ownagefool 7y ago
You don't really have a solution to: Persistent Storage, Pod Security Policies, Admission Controllers (such as checking your deps have been security checked), Resource Utilization or Creating environments (you mentioned it but never specified it). You don't actually understand how k8s does healthProbes. Most of the problems you're leveling at k8s still exist with your technology choices, or any really. An incompetent admin is incompetent whatever the tool. Containers lacking reproducible builds are a bit of a problem, but I doubt it's a problem solved by many/any tools in it's entirety. For the record, I've actually used k8s to run critical national infrastructure for major government services with considerable load, that categorically should not go down. I've also used it as a shared platform for CI/CD in major orgs with lots of seperate delivery teams. It's literally bliss compared to trying to create that with openstack, VMWare or an IaaS provider. You should really try the tool in anger before you dismiss it, because most of your points aren't legit pointing at what k8s gives you. Also, as someone who is a competent admin, who has also been a software developer, having a sysadmin who can barely code trying to debug someone elses at 4am is also a broken model. Having developers involved in support and building of things, is so they can actually suffer the pain of code that isn't reliable, doesn't log or event, doesn't tell you it's started, doesn't have health checks, isn't easy to deploy, etc, etc.
- cookiecaper 7y ago
  > You don't really have a solution to: Persistent Storage, Pod Security Policies, Admission Controllers (such as checking your deps have been security checked), Resource Utilization or Creating environments (you mentioned it but never specified it). Again, I'm not going to draft out a complete architecture for some hypothetical application. The point is that k8s leaves the fundamental questions unsolved, just like non-k8s. This is noteworthy because many people are apparently operating under the belief that k8s will "reduce complexity" by handling these core infrastructure problems transparently and intrinsically, and it doesn't. > Most of the problems you're leveling at k8s still exist with your technology choices, or any really. Right, that's exactly the point. Most people say k8s is worthwhile because they think it's a magic bullet that has self-contained and automatic remediations for core infrastructure concerns. That's because there's no way that the complexity is worthwhile if it doesn't. If you're left with the same basic set of problems regardless, what are you getting by putting k8s in there after all? k8s is probably not the wrong choice 100% of the time. It's just that most people who are jumping on that bandwagon are simply jumping on a bandwagon, and flailing and screaming to everyone else that the bandwagon is a magical land of fairy tales and unicorns. If there is legitimate, real, well-vetted engineering rationale for selecting k8s for a particular use case, it should be selected, of course. Vague statements alluding to its mystic "literally bliss"-inducing powers do not comprise this, despite popular opinion to the contrary. > You should really try the tool in anger before you dismiss it, because most of your points aren't legit pointing at what k8s gives you. I have used it, repeatedly. Granted that the last cluster I ran in prod was a couple of years ago, and I'm sure things have improved in that time. But it doesn't change the fundamental equation of the cost/benefit tradeoff. > having a sysadmin who can barely code trying to debug someone elses at 4am is also a broken model. Agreed, obviously. What does that have to do with Kubernetes?

ownagefool 7y ago

You don't really have a solution to: Persistent Storage, Pod Security Policies, Admission Controllers (such as checking your deps have been security checked), Resource Utilization or Creating environments (you mentioned it but never specified it). You don't actually understand how k8s does healthProbes. Most of the problems you're leveling at k8s still exist with your technology choices, or any really. An incompetent admin is incompetent whatever the tool. Containers lacking reproducible builds are a bit of a problem, but I doubt it's a problem solved by many/any tools in it's entirety. For the record, I've actually used k8s to run critical national infrastructure for major government services with considerable load, that categorically should not go down. I've also used it as a shared platform for CI/CD in major orgs with lots of seperate delivery teams. It's literally bliss compared to trying to create that with openstack, VMWare or an IaaS provider. You should really try the tool in anger before you dismiss it, because most of your points aren't legit pointing at what k8s gives you. Also, as someone who is a competent admin, who has also been a software developer, having a sysadmin who can barely code trying to debug someone elses at 4am is also a broken model. Having developers involved in support and building of things, is so they can actually suffer the pain of code that isn't reliable, doesn't log or event, doesn't tell you it's started, doesn't have health checks, isn't easy to deploy, etc, etc.

cookiecaper 7y ago
> You don't really have a solution to: Persistent Storage, Pod Security Policies, Admission Controllers (such as checking your deps have been security checked), Resource Utilization or Creating environments (you mentioned it but never specified it). Again, I'm not going to draft out a complete architecture for some hypothetical application. The point is that k8s leaves the fundamental questions unsolved, just like non-k8s. This is noteworthy because many people are apparently operating under the belief that k8s will "reduce complexity" by handling these core infrastructure problems transparently and intrinsically, and it doesn't. > Most of the problems you're leveling at k8s still exist with your technology choices, or any really. Right, that's exactly the point. Most people say k8s is worthwhile because they think it's a magic bullet that has self-contained and automatic remediations for core infrastructure concerns. That's because there's no way that the complexity is worthwhile if it doesn't. If you're left with the same basic set of problems regardless, what are you getting by putting k8s in there after all? k8s is probably not the wrong choice 100% of the time. It's just that most people who are jumping on that bandwagon are simply jumping on a bandwagon, and flailing and screaming to everyone else that the bandwagon is a magical land of fairy tales and unicorns. If there is legitimate, real, well-vetted engineering rationale for selecting k8s for a particular use case, it should be selected, of course. Vague statements alluding to its mystic "literally bliss"-inducing powers do not comprise this, despite popular opinion to the contrary. > You should really try the tool in anger before you dismiss it, because most of your points aren't legit pointing at what k8s gives you. I have used it, repeatedly. Granted that the last cluster I ran in prod was a couple of years ago, and I'm sure things have improved in that time. But it doesn't change the fundamental equation of the cost/benefit tradeoff. > having a sysadmin who can barely code trying to debug someone elses at 4am is also a broken model. Agreed, obviously. What does that have to do with Kubernetes?