7 ms·

What does the Kubernetes configuration format offer over configuration management systems like Ansible, Salt, Puppet, Chef, etc?

birdyrooster 6y ago
They are not comparable. You might use ansible, salt, puppet or chef to deploy kubelet, apiserver, etc. You could, barring those with self-love, even deploy Ansible tower on Kubernetes to manage your kubernetes infrastructure.
- deleted 6y ago
  [deleted]
silviogutierrez 6y ago
I'm not intimately familiar with those, but I did a lot of similar things with scripts. As far as I can tell: those are imperative. At least in some areas. Kubernetes is declarative. You mention the end state and it just "figures it out". Mind you, with issues sometimes. All abstractions leak. Note that k8s's adamance about declarative configuration can make you bend over backwards. Example: running a migration script post deploys. Or waiting for other services to start before starting your own. Etc. I think in many ways, those compete with Terraform which is "declarative"-ish. There's very much a state file.
- bcrosby95 6y ago
  I've only ever used cfengine and Ansible, but they are both declarative. Hell, Ansible uses yaml files too. I would be somewhat surprised to find out Puppet and Chef weren't declarative either. Because setting up a system in an imperative fashion is ripe for trouble. You may as well use bash scripts at that point. I've used Ansible for close to 10 years for hobby projects. And setting up my development environment. Give me a freshly installed Ubuntu laptop, and I can have my development environment 100% setup with a single command.
  - james-mcelwain 6y ago
    Even though Ansible is declarative in spirit, in practice I feel like a lot of playbooks just read like imperative code.
    - fastball 6y ago
      I use Ansible for managing my infra, and the only time my playbooks look imperative is when I execute a shell script or similar, which is about 5% of total commands in my playbooks.
      - takeda 6y ago
        One way to test if your playbook is declarative is try to rearrange the states and have them in different order. If the playbook breaks with different order it is imperative.
  - silviogutierrez 6y ago
    Check out nix for actual development environments. Huge fan of that as well. I can buy a new laptop and be back to 100% in a few minutes. Though the amount of time I spent learning how to get there far exceeds any time savings. Ever.
    - sam_lowry_ 6y ago
      Hm... I just dd my hard disk to the new one and launch gparted to resize. A few years ago I even bothered to have two EFI loaders: one for amd an one intel, in case I want to change architecture as well.
    - open-paren 6y ago
      I've been testing out nix and I haven't found out how to install packages in a declarative way yet. Using "nix-env -iA <whatever>` seems really imperative. How are you doing that? Do you use something like home-manager, or do you just define a default.nix and then nix-shell it whenever you need something?
bosswipe 6y ago
According to the article the advantage of Kubernetes is that you're not writing code like you are with Puppet and Chef. You're writing YAML files.
p_l 6y ago
Having extensively used Chef and K8s, the difference is that they try to deal with chaos in unmanaged way (Puppet is the closest to "managed"), but when dealing with wild chaos you lack many ways of enforcing the order. Plus they don't really do multi-server computation of resources. What k8s brings to the table is a level of standardization. It's the difference between bringing some level of robotics to manual loading and unloading of classic cargo ships, vs. the fully automated containerized ports. With k8s, you get structure where you can wrap individual program's idiosyncracies into a container that exposes standard interface. This standard interface allows you to then easily drop it into server, with various topologies, resources, networking etc. handled through common interfaces. I said that for a long time before, but recently I got to understand just how much work k8s can "take away" when I foolishly said "eh, it's only one server, I will run this the classic way. Then I spent 5 days on something that could be handled within an hour on k8s, because k8s virtualized away HTTP reverse proxies, persistent storage, and load balancing in general. Now I'm thinking of deploying k8s at home, not to learn, but because I know it's easier for me to deploy nextcloud, or an ebook catalog, or whatever, using k8s than by setting up more classical configuration management system and deal with inevitable drift over time.
- naringas 6y ago
  > Now I'm thinking of deploying k8s at home, not to learn, but because I know it's easier for me to deploy nextcloud, or an ebook catalog can't you do that just with containers?
  - aequitas 6y ago
    But what do you use to manage those containers and surrounding infra (networking, proxies, etc)? I've been down the route of using Puppet for managing Docker containers on existing systems, Ansible, Terraform, Nomad/Consul. But in the end it all is just tying different solutions together to make it work. Kubernetes (in the form of K3s or a other lightweight implementation) just works for me, even in a single server setup. I barely have to worry about the OS layer, I just flash K3s to a disk and only have to talk to the Kubernetes API to apply declarative configurations. Only things I'm sometimes still need the OS layer for is networking, firewall or hardening of the base OS. But that configuration is mostly static anyways and I'm sure I will fine some operators for that to manage then through the Kubernetes API as IaC if I really need to.
dustym 6y ago
For certain things like layer 4 and layer 7 routing or firewall policies, health checking and failover, network-attached volumes, etc you have to choose software and configure it on top of getting that configuration in that tooling. So you are doing kernel or iptables or nginx or monit/supervisord configurations and so on. But basic versions of these things are provided by Kubernetes natively and can be declared in a way that is divorced from configuring the underlying software. So you just learn how to configure these broader concepts as services or ingresses or network policies, etc, and don't worry about the underlying implementations. It's pretty nice actually.
kissgyorgy 6y ago
Ansible configuration is imperative (you need to run notebooks in order) but Kubernetes YAML is declarative. That alone is a huge difference!
2kewl4skewl 6y ago
Containers are the big difference. Kubernetes is one way to deploy containers. Configuration systems like Ansible/Salt/Puppet/Chef/etc are another way to deploy containers. Kubernetes also makes it possible to dynamically scale your workload. But so does Auto Scaling Groups (AWS terminology) and GCP/Azure equivalents. The reality is that 99% of users don't actually need Kubernetes. It introduces a huge amount of complexity, overhead, and instability for no benefit in most cases. The tech industry is highly trend driven. There is a lot of cargo culting. People want to build their resumes. They like novelty. Many people incorrectly believe that Kubernetes is the way to deploy containers. And they (and their employers) suffer for it. Most users would be far better off using boring statically deployed containers from a configuration management system. Auto-scaled when required. This can also be entirely infrastructure-as-code compliant. Containers are the real magic. But somehow people confused Kubernetes as a replacement for Docker containers, when it was actually a replacement for Docker's orchestration framework: Docker Swarm. In fact, Kubernetes is a very dangerous chainsaw that most people are using to whittle in their laps.
- geggam 6y ago
  >In fact, Kubernetes is a very dangerous chainsaw that most people are using to whittle in their laps. So many people miss this. k8s is a very complex system and the talent it takes to manage it well, rare. Extremely rare.
  - geggam 6y ago
    At some point I think I will setup an email account so I can offer to interview those downvoters 1st Question : Define k8s network, in detail, with all of the services and a set of services IF you make it out of that one and the follow ups we can move on to the rest
  - adolfojp 6y ago
    Expectations have been set unrealistically high and online communities like this one make matters worse. Big players with dedicated devops teams use Kubernetes all the time so why shouldn't I? It's only a matter of time before "Hello world" tutorials include a chapter on container orchestration. So we end up with a plethora of full stack developers who can barely keep up with their current development stacks willfully deploying their software on systems that they're just barely competent with. I know this because I almost deployed a side project with Kubernetes because it was expected of me despite the fact that being mediocre at it was the best that I could hope to become and that's an easy way to chop off a leg or three.
MuffinFlavored 6y ago
What about Terraform?
tapoxi 6y ago
I've been using Kubernetes exclusively for the past two years after coming from a fairly large Saltstack shop. I think traditional configuration management is flawed. Configuration drift _will_ happen because something, somewhere, will do something you or the formula/module/playbook didn't account for. A Dockerfile builds the world from (almost) scratch and forces the runtime environment to be stateless. A CM tool constantly tries to shape the world to its image. Kubernetes isn't a silver bullet of course, there will be applications where running it in containers adds unnecessary complexity, and those are best run in a VM managed by a CM tool. I'd argue using k8s is safe default for deploying new applications going forward.
chrissoundz 6y ago
It's not just the configuration format. There is a whole 'Kubernetes runtime' (what they call the 'control loops' aka 'controllers') that runs 24/7 watching the configuration live and making appropriate changes. Unlike Ansible (and I suspect the others) where it's really only more of a 'run once' type of thing... And sometimes if you try running it a second time it won't even succeed.
- closeparen 6y ago
  Ansible is a little special in how imperative it is, a better comparison is Puppet which is intended to do periodic "convergence" runs, although these are more typically hourly or daily than continuous.

birdyrooster 6y ago

They are not comparable. You might use ansible, salt, puppet or chef to deploy kubelet, apiserver, etc. You could, barring those with self-love, even deploy Ansible tower on Kubernetes to manage your kubernetes infrastructure.

deleted 6y ago
[deleted]

silviogutierrez 6y ago

I'm not intimately familiar with those, but I did a lot of similar things with scripts. As far as I can tell: those are imperative. At least in some areas. Kubernetes is declarative. You mention the end state and it just "figures it out". Mind you, with issues sometimes. All abstractions leak. Note that k8s's adamance about declarative configuration can make you bend over backwards. Example: running a migration script post deploys. Or waiting for other services to start before starting your own. Etc. I think in many ways, those compete with Terraform which is "declarative"-ish. There's very much a state file.

bcrosby95 6y ago
I've only ever used cfengine and Ansible, but they are both declarative. Hell, Ansible uses yaml files too. I would be somewhat surprised to find out Puppet and Chef weren't declarative either. Because setting up a system in an imperative fashion is ripe for trouble. You may as well use bash scripts at that point. I've used Ansible for close to 10 years for hobby projects. And setting up my development environment. Give me a freshly installed Ubuntu laptop, and I can have my development environment 100% setup with a single command.
- james-mcelwain 6y ago
  Even though Ansible is declarative in spirit, in practice I feel like a lot of playbooks just read like imperative code.
  - fastball 6y ago
    I use Ansible for managing my infra, and the only time my playbooks look imperative is when I execute a shell script or similar, which is about 5% of total commands in my playbooks.
    - takeda 6y ago
      One way to test if your playbook is declarative is try to rearrange the states and have them in different order. If the playbook breaks with different order it is imperative.
- silviogutierrez 6y ago
  Check out nix for actual development environments. Huge fan of that as well. I can buy a new laptop and be back to 100% in a few minutes. Though the amount of time I spent learning how to get there far exceeds any time savings. Ever.
  - sam_lowry_ 6y ago
    Hm... I just dd my hard disk to the new one and launch gparted to resize. A few years ago I even bothered to have two EFI loaders: one for amd an one intel, in case I want to change architecture as well.
  - open-paren 6y ago
    I've been testing out nix and I haven't found out how to install packages in a declarative way yet. Using "nix-env -iA <whatever>` seems really imperative. How are you doing that? Do you use something like home-manager, or do you just define a default.nix and then nix-shell it whenever you need something?

bosswipe 6y ago

According to the article the advantage of Kubernetes is that you're not writing code like you are with Puppet and Chef. You're writing YAML files.

p_l 6y ago

Having extensively used Chef and K8s, the difference is that they try to deal with chaos in unmanaged way (Puppet is the closest to "managed"), but when dealing with wild chaos you lack many ways of enforcing the order. Plus they don't really do multi-server computation of resources. What k8s brings to the table is a level of standardization. It's the difference between bringing some level of robotics to manual loading and unloading of classic cargo ships, vs. the fully automated containerized ports. With k8s, you get structure where you can wrap individual program's idiosyncracies into a container that exposes standard interface. This standard interface allows you to then easily drop it into server, with various topologies, resources, networking etc. handled through common interfaces. I said that for a long time before, but recently I got to understand just how much work k8s can "take away" when I foolishly said "eh, it's only one server, I will run this the classic way. Then I spent 5 days on something that could be handled within an hour on k8s, because k8s virtualized away HTTP reverse proxies, persistent storage, and load balancing in general. Now I'm thinking of deploying k8s at home, not to learn, but because I know it's easier for me to deploy nextcloud, or an ebook catalog, or whatever, using k8s than by setting up more classical configuration management system and deal with inevitable drift over time.

naringas 6y ago
> Now I'm thinking of deploying k8s at home, not to learn, but because I know it's easier for me to deploy nextcloud, or an ebook catalog can't you do that just with containers?
- aequitas 6y ago
  But what do you use to manage those containers and surrounding infra (networking, proxies, etc)? I've been down the route of using Puppet for managing Docker containers on existing systems, Ansible, Terraform, Nomad/Consul. But in the end it all is just tying different solutions together to make it work. Kubernetes (in the form of K3s or a other lightweight implementation) just works for me, even in a single server setup. I barely have to worry about the OS layer, I just flash K3s to a disk and only have to talk to the Kubernetes API to apply declarative configurations. Only things I'm sometimes still need the OS layer for is networking, firewall or hardening of the base OS. But that configuration is mostly static anyways and I'm sure I will fine some operators for that to manage then through the Kubernetes API as IaC if I really need to.

dustym 6y ago

For certain things like layer 4 and layer 7 routing or firewall policies, health checking and failover, network-attached volumes, etc you have to choose software and configure it on top of getting that configuration in that tooling. So you are doing kernel or iptables or nginx or monit/supervisord configurations and so on. But basic versions of these things are provided by Kubernetes natively and can be declared in a way that is divorced from configuring the underlying software. So you just learn how to configure these broader concepts as services or ingresses or network policies, etc, and don't worry about the underlying implementations. It's pretty nice actually.

kissgyorgy 6y ago

Ansible configuration is imperative (you need to run notebooks in order) but Kubernetes YAML is declarative. That alone is a huge difference!

2kewl4skewl 6y ago

Containers are the big difference. Kubernetes is one way to deploy containers. Configuration systems like Ansible/Salt/Puppet/Chef/etc are another way to deploy containers. Kubernetes also makes it possible to dynamically scale your workload. But so does Auto Scaling Groups (AWS terminology) and GCP/Azure equivalents. The reality is that 99% of users don't actually need Kubernetes. It introduces a huge amount of complexity, overhead, and instability for no benefit in most cases. The tech industry is highly trend driven. There is a lot of cargo culting. People want to build their resumes. They like novelty. Many people incorrectly believe that Kubernetes is the way to deploy containers. And they (and their employers) suffer for it. Most users would be far better off using boring statically deployed containers from a configuration management system. Auto-scaled when required. This can also be entirely infrastructure-as-code compliant. Containers are the real magic. But somehow people confused Kubernetes as a replacement for Docker containers, when it was actually a replacement for Docker's orchestration framework: Docker Swarm. In fact, Kubernetes is a very dangerous chainsaw that most people are using to whittle in their laps.

geggam 6y ago
>In fact, Kubernetes is a very dangerous chainsaw that most people are using to whittle in their laps. So many people miss this. k8s is a very complex system and the talent it takes to manage it well, rare. Extremely rare.
- geggam 6y ago
  At some point I think I will setup an email account so I can offer to interview those downvoters 1st Question : Define k8s network, in detail, with all of the services and a set of services IF you make it out of that one and the follow ups we can move on to the rest
- adolfojp 6y ago
  Expectations have been set unrealistically high and online communities like this one make matters worse. Big players with dedicated devops teams use Kubernetes all the time so why shouldn't I? It's only a matter of time before "Hello world" tutorials include a chapter on container orchestration. So we end up with a plethora of full stack developers who can barely keep up with their current development stacks willfully deploying their software on systems that they're just barely competent with. I know this because I almost deployed a side project with Kubernetes because it was expected of me despite the fact that being mediocre at it was the best that I could hope to become and that's an easy way to chop off a leg or three.

MuffinFlavored 6y ago

What about Terraform?

tapoxi 6y ago

I've been using Kubernetes exclusively for the past two years after coming from a fairly large Saltstack shop. I think traditional configuration management is flawed. Configuration drift _will_ happen because something, somewhere, will do something you or the formula/module/playbook didn't account for. A Dockerfile builds the world from (almost) scratch and forces the runtime environment to be stateless. A CM tool constantly tries to shape the world to its image. Kubernetes isn't a silver bullet of course, there will be applications where running it in containers adds unnecessary complexity, and those are best run in a VM managed by a CM tool. I'd argue using k8s is safe default for deploying new applications going forward.

chrissoundz 6y ago

It's not just the configuration format. There is a whole 'Kubernetes runtime' (what they call the 'control loops' aka 'controllers') that runs 24/7 watching the configuration live and making appropriate changes. Unlike Ansible (and I suspect the others) where it's really only more of a 'run once' type of thing... And sometimes if you try running it a second time it won't even succeed.

closeparen 6y ago
Ansible is a little special in how imperative it is, a better comparison is Puppet which is intended to do periodic "convergence" runs, although these are more typically hourly or daily than continuous.