4 ms·

> You ended up with lots of magic an internal domain knowledge about how to do things, and often had to get clever to get the result you wanted. How does Kuber

by binoct 5y ago

> You ended up with lots of magic an internal domain knowledge about how to do things, and often had to get clever to get the result you wanted. How does Kubernetes not also result in the same? I'm not questioning plenty of improvements and the somewhat different domain it brings to the table overall but genuinely curious. From what I've seen any tool designed to integrate and manage complex, disparate systems is going to end up with lots special cases, domain knowledge, and require digging into implementation details on a regular basis for anything other than absolute common-case uses.

pm90 5y ago
One of the best things about K8s is its pretty stable API. We use GKE and the upgrades happen without issues 99% of the time. The other benefit that k8s got for "free" was with containers: developers could stuff in whatever environment, software and configs they wanted inside the container. This removed most of the "my app broke because some shared lib didn't install properly" sorts of issues. This separation b/w software configuration and application configuration is extremely powerful.
waheoo 5y ago
Who said it doesn't? Look at its schema it's horrific. The reason it's popular is that it's backed by Google and solves problems Docker the company couldn't. That doesn't mean k8s is by any means not suffering the same problems of being a dsl that should be something else. It also doesn't mean k8s won't suffer the same weirdness between platforms (AWS vs GCE vs GKE).
pat2man 5y ago
Kubernetes doesn't attempt to be an API that can solve any orchestration issue, it is an abstraction on the underlying architecture. It's a lot easier to be backwards compatible here (and if you are not, create an operator that is). Now whether its the right abstraction is a matter of debate. But ultimately like VMs or containers, its so far proven to be a valuable tool, at least for medium sized companies.
- saargrin 5y ago
  and then you use something like helm to orchestrate and end up being in the same place where you left ansible: lots of environment specific magic and flow-breaking with every upgrade of anything same with cloud envs and terraform
gouggoug 5y ago
> How does Kubernetes not also result in the same? Here's the difference between kubernetes and ansible (and alikes) (and I'm over-simplifying a bit of course, but the idea is there) * With kubernetes, you write down the state you want your infrastructure to be in, and k8s figures out how to get there * With ansible, you write down how to get to the state you want and ansible runs those steps, and hopefully they end up in the same state. The issue is that if you run the same set of steps on a Debian linux distribution, and on a Ubuntu distribution, well, they might end up in a similar state, but they actually aren't; for starters: one is debian, the other is unbuntu.
- dale_glass 5y ago
  Ansible does try to do the 'write down the state you want your infrastructure to be in' thing. Eg, - name: "Ensure sshd is started and enabled on boot" systemd: state: started enabled: yes name: sshd You're not saying 'enable the sshd service', but 'the desired state for sshd is started and enabled'. But this is not perfect, because it's easy to eventually end up with imperative commands, like "run this script", or "Copy this file".
  - kspacewalk2 5y ago
    That does not seem avoidable. I have never used kubernetes, but this seems highly suspect: >With kubernetes, you write down the state you want your infrastructure to be in, and k8s figures out how to get there My guess is that this devolves to "run this script" or "copy this file" in practice as well. Or it just becomes tautological, i.e. "here's the state I want the system to be in (and that means run this script and then copy this file over here, and don't forget to symlink it over there)". Am I wrong?
    - scrollaway 5y ago
      Yeah you're (mostly) incorrect. I recommend looking into terraform and how it handles state, that will give you a better idea of how it's possible to do infrastructure in a "this is what i want, figure out how to get there" way. Terraform isn't the best implementation of it; k8s has full control over the infra so it's even more powerful in that regard.
bostonsre 5y ago
Yea, I think with both ansible (and stuff like puppet) and kubernetes they are both powerful tools that are pretty leaky abstractions. They are great for sharing and implementing recipes, but that abstraction layer makes it a little more difficult to become a master chef and understand/control exactly what's happening. They are awesome to have in your toolbox, but I still find it incredibly useful to have a tool like python in there as well.
colechristensen 5y ago
I’m not saying kube really fixes all of these problems or is a better solution, but it exists as a response to a crappy solution. in other words one problematic solution replaced another problematic solution. Stay in the field long enough and you see a pendulum where one kind of solution with problems is replaced by a different kind of solution with different problems... and then it swings back and a new general goes back to a new iteration of the first kind of solution and gets to rediscover the old problems again, back and forth over and over.

by binoct 5y ago

pm90 5y ago

One of the best things about K8s is its pretty stable API. We use GKE and the upgrades happen without issues 99% of the time. The other benefit that k8s got for "free" was with containers: developers could stuff in whatever environment, software and configs they wanted inside the container. This removed most of the "my app broke because some shared lib didn't install properly" sorts of issues. This separation b/w software configuration and application configuration is extremely powerful.

waheoo 5y ago

Who said it doesn't? Look at its schema it's horrific. The reason it's popular is that it's backed by Google and solves problems Docker the company couldn't. That doesn't mean k8s is by any means not suffering the same problems of being a dsl that should be something else. It also doesn't mean k8s won't suffer the same weirdness between platforms (AWS vs GCE vs GKE).

pat2man 5y ago

Kubernetes doesn't attempt to be an API that can solve any orchestration issue, it is an abstraction on the underlying architecture. It's a lot easier to be backwards compatible here (and if you are not, create an operator that is). Now whether its the right abstraction is a matter of debate. But ultimately like VMs or containers, its so far proven to be a valuable tool, at least for medium sized companies.

saargrin 5y ago
and then you use something like helm to orchestrate and end up being in the same place where you left ansible: lots of environment specific magic and flow-breaking with every upgrade of anything same with cloud envs and terraform

gouggoug 5y ago

> How does Kubernetes not also result in the same? Here's the difference between kubernetes and ansible (and alikes) (and I'm over-simplifying a bit of course, but the idea is there) * With kubernetes, you write down the state you want your infrastructure to be in, and k8s figures out how to get there * With ansible, you write down how to get to the state you want and ansible runs those steps, and hopefully they end up in the same state. The issue is that if you run the same set of steps on a Debian linux distribution, and on a Ubuntu distribution, well, they might end up in a similar state, but they actually aren't; for starters: one is debian, the other is unbuntu.

dale_glass 5y ago
Ansible does try to do the 'write down the state you want your infrastructure to be in' thing. Eg, - name: "Ensure sshd is started and enabled on boot" systemd: state: started enabled: yes name: sshd You're not saying 'enable the sshd service', but 'the desired state for sshd is started and enabled'. But this is not perfect, because it's easy to eventually end up with imperative commands, like "run this script", or "Copy this file".
- kspacewalk2 5y ago
  That does not seem avoidable. I have never used kubernetes, but this seems highly suspect: >With kubernetes, you write down the state you want your infrastructure to be in, and k8s figures out how to get there My guess is that this devolves to "run this script" or "copy this file" in practice as well. Or it just becomes tautological, i.e. "here's the state I want the system to be in (and that means run this script and then copy this file over here, and don't forget to symlink it over there)". Am I wrong?
  - scrollaway 5y ago
    Yeah you're (mostly) incorrect. I recommend looking into terraform and how it handles state, that will give you a better idea of how it's possible to do infrastructure in a "this is what i want, figure out how to get there" way. Terraform isn't the best implementation of it; k8s has full control over the infra so it's even more powerful in that regard.

bostonsre 5y ago

Yea, I think with both ansible (and stuff like puppet) and kubernetes they are both powerful tools that are pretty leaky abstractions. They are great for sharing and implementing recipes, but that abstraction layer makes it a little more difficult to become a master chef and understand/control exactly what's happening. They are awesome to have in your toolbox, but I still find it incredibly useful to have a tool like python in there as well.

colechristensen 5y ago

I’m not saying kube really fixes all of these problems or is a better solution, but it exists as a response to a crappy solution. in other words one problematic solution replaced another problematic solution. Stay in the field long enough and you see a pendulum where one kind of solution with problems is replaced by a different kind of solution with different problems... and then it swings back and a new general goes back to a new iteration of the first kind of solution and gets to rediscover the old problems again, back and forth over and over.