6 ms·

Kubernetes is what happens when you need to support everyone's wants and desires within the core platform. The abstraction facade breaks and ends up exposing a

Kubernetes is what happens when you need to support everyone's wants and desires within the core platform. The abstraction facade breaks and ends up exposing all of the underlying pieces because someone needs feature X. So much of Kubernetes' complexity is YAGNI (for most users). Kubernetes 2.0 should be a boring pod scheduler with some RBAC around it. Let folks swap out the abstractions if they need it instead of having everything so tightly coupled within the core platform.

echelon 1y ago
Kubernetes doesn't need a flipping package manager or charts. It needs to do one single job well: workload scheduling. Kubernetes clusters shouldn't be bespoke and weird with behaviors that change based on what flavor of plugins you added. That is antithetical to the principal of the workloads you're trying to manage. You should be able to headshot the whole thing with ease. Service discovery is just one of many things that should be a different layer.
- KaiserPro 1y ago
  > Service discovery is just one of many things that should be a different layer. hard agree. Its like jenkins, good idea, but its not portable.
  - 12_throw_away 1y ago
    > Its like jenkins Having regretfully operated both k8s and Jenkins, I fully agree with this, they have some deep DNA in common.
sitkack 1y ago
Kubernetes is when you want to sell complexity because complexity makes money and and naturally gets you vendor lockin even while being ostensibly vendor neutral. Never interrupt the customer while they are foot gunning themselves. Swiss Army Buggy Whips for Everyone!
- wredcoll 1y ago
  Not really. Kubernetes is still wildly simpler than what came before, especially accounting for the increased capabilities.
  - cogman10 1y ago
    Yup. Having migrated from a puppet + custom scripts environment and terraform + custom scripts. K8S is a breath of fresh air. I get that it's not for everyone, I'd not recommend it for everyone. But once you start getting a pretty diverse ecosystem of services, k8s solves a lot of problems while being pretty cheap. Storage is a mess, though, and something that really needs to be addressed. I typically recommend people wanting persistence to not use k8s.
    - mdaniel 1y ago
      > Storage is a mess, though, and something that really needs to be addressed. I typically recommend people wanting persistence to not use k8s. I have actually come to wonder if this is actually an AWS problem, and not a Kubernetes problem. I mention this because the CSI controllers seem to behave sanely, but they are only as good as the requests being fulfilled by the IaaS control plane. I secretly suspect that EBS just wasn't designed for such a hot-swap world Now, I posit this because I haven't had to run clusters in Azure nor GCP to know if my theory has legs I guess the counter-experiment would be to forego the AWS storage layer and try Ceph or Longhorn but no company I've ever worked at wants to blaze trails about that, so they just build up institutional tribal knowledge about treating PVCs with kid gloves
      - wredcoll 1y ago
        Honestly this just feels like kubernetes just solving the easy problems and ignoring the hard bits. You notice the pattern a lot after a certain amount of time watching new software being built.
        mdaniel 1y ago
selcuka 1y ago
> Let folks swap out the abstractions if they need it instead of having everything so tightly coupled within the core platform. Sure, but then one of those third party products (say, X) will catch up, and everyone will start using it. Then job ads will start requiring "10 year of experience in X". Then X will replace the core orchestrator (K8s) with their own implementation. Then we'll start seeing comments like "X is a horribly complex, bloated platform which should have been just a boring orchestrator" on HN.

echelon 1y ago

Kubernetes doesn't need a flipping package manager or charts. It needs to do one single job well: workload scheduling. Kubernetes clusters shouldn't be bespoke and weird with behaviors that change based on what flavor of plugins you added. That is antithetical to the principal of the workloads you're trying to manage. You should be able to headshot the whole thing with ease. Service discovery is just one of many things that should be a different layer.

KaiserPro 1y ago
> Service discovery is just one of many things that should be a different layer. hard agree. Its like jenkins, good idea, but its not portable.
- 12_throw_away 1y ago
  > Its like jenkins Having regretfully operated both k8s and Jenkins, I fully agree with this, they have some deep DNA in common.

sitkack 1y ago

Kubernetes is when you want to sell complexity because complexity makes money and and naturally gets you vendor lockin even while being ostensibly vendor neutral. Never interrupt the customer while they are foot gunning themselves. Swiss Army Buggy Whips for Everyone!

wredcoll 1y ago
Not really. Kubernetes is still wildly simpler than what came before, especially accounting for the increased capabilities.
- cogman10 1y ago
  Yup. Having migrated from a puppet + custom scripts environment and terraform + custom scripts. K8S is a breath of fresh air. I get that it's not for everyone, I'd not recommend it for everyone. But once you start getting a pretty diverse ecosystem of services, k8s solves a lot of problems while being pretty cheap. Storage is a mess, though, and something that really needs to be addressed. I typically recommend people wanting persistence to not use k8s.
  - mdaniel 1y ago
    > Storage is a mess, though, and something that really needs to be addressed. I typically recommend people wanting persistence to not use k8s. I have actually come to wonder if this is actually an AWS problem, and not a Kubernetes problem. I mention this because the CSI controllers seem to behave sanely, but they are only as good as the requests being fulfilled by the IaaS control plane. I secretly suspect that EBS just wasn't designed for such a hot-swap world Now, I posit this because I haven't had to run clusters in Azure nor GCP to know if my theory has legs I guess the counter-experiment would be to forego the AWS storage layer and try Ceph or Longhorn but no company I've ever worked at wants to blaze trails about that, so they just build up institutional tribal knowledge about treating PVCs with kid gloves
    - wredcoll 1y ago
      Honestly this just feels like kubernetes just solving the easy problems and ignoring the hard bits. You notice the pattern a lot after a certain amount of time watching new software being built.
      - mdaniel 1y ago

selcuka 1y ago

> Let folks swap out the abstractions if they need it instead of having everything so tightly coupled within the core platform. Sure, but then one of those third party products (say, X) will catch up, and everyone will start using it. Then job ads will start requiring "10 year of experience in X". Then X will replace the core orchestrator (K8s) with their own implementation. Then we'll start seeing comments like "X is a horribly complex, bloated platform which should have been just a boring orchestrator" on HN.