4 ms·

Can you clarify what you mean by "leaks implementation details everywhere"? I like to think of kubernetes as a big orchestration platform that you can choose t

Can you clarify what you mean by "leaks implementation details everywhere"? I like to think of kubernetes as a big orchestration platform that you can choose to use what you need. If an ingress and pods work then use that, otherwise extend an throw an operator up for what you need (it likely already exists). Cilium for instance is great for that, so is Istio and the like. They aren't hard you just have to understand networking... which is nearly the same energy of running it on another orchestration tool or raw on a network device.

weitendorf 2y ago
You shouldn’t have to think about all the implementation details of your deployment target. There shouldn’t be platform engineers or Kubernetes experts. Nobody should be writing YAML, or getting paid to set up Istio. Nobody should have to learn the Kubernetes architecture or know about the kubelet or EBPF. The tools should be simple enough with good defaults that application developers just click a button and have their code run somewhere. Right now platform engineers fill that gap, because the underlying tech doesn’t. IMO you are thinking too much like an engineer if you are saying you “just have to understand networking”. Why? It’s always better when the problem gets solved in a way that allows you to not think of it too much. Right now SREs and the platform team do that for application developers because Kubernetes only does it halfway. Infrastructure/devops/SRE is a pure means to an end of getting actual applications (the ultimate source of all the value in software) to run. It’s an obstacle. Right now the obstacle is bigger than it needs to be
- stackskipton 2y ago
  Ops here, In a simple Kubernetes environment, you don't have to know networking. However, few environments are simple and abstracting X away becomes extremely difficult job once business requirements collide with abstraction. Obstacle is big most of time because most applications are not easy to run. Most DevOps mostly came around because Devs flinging balls of mud over the wall and landing on the Ops side with a splat and then screaming when we can't build nets to catch the mud ball and Ops is covered with mud. Sure it failed just like DevSecOps fails because most of time, Devs don't care about anything other than closing Jira tickets and going home.
  - osigurdson 2y ago
    I think that devs really should have a decent understanding of Kubernetes. It is essentially the operating system for any app that needs more than one computer.
- klooney 2y ago
  You don't need any of that, EKS works out of the box with Fargate. But companies don't want that, they want to support EKS and data centers, which means supporting the "implementation" side of all of the interfaces, which means getting down into the details. The real problem is that every platform team, deep down, wants to rewrite EKS and they often do, which I would describe as "a giant money pit".
- Whitespace 2y ago
  Is it fair to say that you are in favor of Heroku, fly.io, ECS, etc?
  - weitendorf 2y ago
    I think those products are much better than k8s for many use cases, but I don’t think that anybody has really delivered the right thing in that space yet.
- llama052 2y ago
  So there should be magic on every layer except for the application? That will never happen, the only thing you can do is pay someone like heroku to take care of that for you. Or if your project is small and plain enough you run “serverless” which is just routing to another platform team, as I’m sure you’re familiar. It’s complicated because there’s a lot that goes on, kubernetes or not.
  - weitendorf 2y ago
    You literally use magic like that all the time with Linux, compilers, language runtimes. Magic is perfectly good when it works. Being able to do anything you want with a quick “presto” is amazing. Nobody should have to learn arcane spells, study the ancient tomes, and communicate with beings of the ethereal plane - somebody just needs to figure out the next “presto”. And historically speaking, usually somebody does when there’s an incentive to do so. In other words - Serverless is another platform team in the same way Linux is another OS team or Java is another language team. And for 99.9% of companies a language team or OS team would be absurd. There’s a big incentive for platform work to go the same way.
nunez 2y ago
Just look at the release notes for every major release. They focus on what's new within Kubernetes architecture instead of what's new that benefits users. Things like having to remove finalizers appended onto resources when you delete them and they hang. Damn near everything about Custom Resources. This isn't a dig at Kubernetes; I love using it. However, I agree that it leaks implementation everywhere.

weitendorf 2y ago

You shouldn’t have to think about all the implementation details of your deployment target. There shouldn’t be platform engineers or Kubernetes experts. Nobody should be writing YAML, or getting paid to set up Istio. Nobody should have to learn the Kubernetes architecture or know about the kubelet or EBPF. The tools should be simple enough with good defaults that application developers just click a button and have their code run somewhere. Right now platform engineers fill that gap, because the underlying tech doesn’t. IMO you are thinking too much like an engineer if you are saying you “just have to understand networking”. Why? It’s always better when the problem gets solved in a way that allows you to not think of it too much. Right now SREs and the platform team do that for application developers because Kubernetes only does it halfway. Infrastructure/devops/SRE is a pure means to an end of getting actual applications (the ultimate source of all the value in software) to run. It’s an obstacle. Right now the obstacle is bigger than it needs to be

stackskipton 2y ago
Ops here, In a simple Kubernetes environment, you don't have to know networking. However, few environments are simple and abstracting X away becomes extremely difficult job once business requirements collide with abstraction. Obstacle is big most of time because most applications are not easy to run. Most DevOps mostly came around because Devs flinging balls of mud over the wall and landing on the Ops side with a splat and then screaming when we can't build nets to catch the mud ball and Ops is covered with mud. Sure it failed just like DevSecOps fails because most of time, Devs don't care about anything other than closing Jira tickets and going home.
- osigurdson 2y ago
  I think that devs really should have a decent understanding of Kubernetes. It is essentially the operating system for any app that needs more than one computer.
klooney 2y ago
You don't need any of that, EKS works out of the box with Fargate. But companies don't want that, they want to support EKS and data centers, which means supporting the "implementation" side of all of the interfaces, which means getting down into the details. The real problem is that every platform team, deep down, wants to rewrite EKS and they often do, which I would describe as "a giant money pit".
Whitespace 2y ago
Is it fair to say that you are in favor of Heroku, fly.io, ECS, etc?
- weitendorf 2y ago
  I think those products are much better than k8s for many use cases, but I don’t think that anybody has really delivered the right thing in that space yet.
llama052 2y ago
So there should be magic on every layer except for the application? That will never happen, the only thing you can do is pay someone like heroku to take care of that for you. Or if your project is small and plain enough you run “serverless” which is just routing to another platform team, as I’m sure you’re familiar. It’s complicated because there’s a lot that goes on, kubernetes or not.
- weitendorf 2y ago
  You literally use magic like that all the time with Linux, compilers, language runtimes. Magic is perfectly good when it works. Being able to do anything you want with a quick “presto” is amazing. Nobody should have to learn arcane spells, study the ancient tomes, and communicate with beings of the ethereal plane - somebody just needs to figure out the next “presto”. And historically speaking, usually somebody does when there’s an incentive to do so. In other words - Serverless is another platform team in the same way Linux is another OS team or Java is another language team. And for 99.9% of companies a language team or OS team would be absurd. There’s a big incentive for platform work to go the same way.

nunez 2y ago

Just look at the release notes for every major release. They focus on what's new within Kubernetes architecture instead of what's new that benefits users. Things like having to remove finalizers appended onto resources when you delete them and they hang. Damn near everything about Custom Resources. This isn't a dig at Kubernetes; I love using it. However, I agree that it leaks implementation everywhere.