6 ms·

Kubernetes is just another layer on top of Docker. It can use other container execution environments, but Kubernetes itself doesn't manage the container runtime

Kubernetes is just another layer on top of Docker. It can use other container execution environments, but Kubernetes itself doesn't manage the container runtime; it just orchestrates containers at a higher level. If you're running Kubernetes, you're almost always still running Docker.

WestCoastJustin 6y ago
Yeah, funny how it all worked out. cgroups/namespaces powered simple LXC containers. Docker comes along and makes a nice workflow and package management system (wrapping cgroups/namespaces). Kubernetes comes along and makes a nice workflow/cluster management layer (wrapping Docker). Cloud providers come along and make a nice Kubernetes management layer (wrapping Kubernetes). Pretty crazy to see the evolution over the past few years. This tech has completely changed the sysadmin landscape/job descriptions and sort of threw tons of gas the whole devops movement. Disclaimer: I worked at both Docker & Google. Although not on this tech specifically. Opinions are my own here.
- tyingq 6y ago
  "threw tons of gas on the whole devops movement" I'm happy to find that I agree with you on at least one area of this new ecosystem :)
  - 0xbadcafebee 6y ago
    This is literally my job and I kind of hate how unnecessarily complex it all is. Not in the sense of "distributed systems are hard", but that we don't actually need the onion to have 20 layers, we really only need 2 or 3. The reason we need an orchestration system for our orchestration system, is the orchestration system is a snowflake. We need an orchestration system because the container system is a snowflake. We need a container system because containers are a snowflake. None of it is really standard or easy to implement, because none of it was designed in the spirit of Unix. It was all just some random companies who threw some crap together so they could start making money selling ads or hosting services. (I don't mean cgroups, I mean the tools that use them, although cgroups are kind of warty in themselves) All most people need to do is run a couple processes on a couple hosts, send packets between services (and networks), and have something start/stop/restart those processes. You can have all of that with, say, systemd (ugh), or three different standard interfaces that any program can use with standard libraries/ABIs. Notice I didn't say "with 5 different daemons running on 3 hosts that need complex configuration, constant maintenance, and a migration effort every 3 months". The Open Container Initiative seems close to getting the first thing done. If we get the other two standardized, we can lose a whole bunch of the onion layers. Consequently a lot of people will need to find a new way to make money, because there'll be not nearly as much need to pay people to deal with the onion.
    - baq 6y ago
      The way you described it one has to wonder what would the tech landscape look like had Plan 9 received adoption instead of Linux, so you didn’t have to invent all those layers of complexity to manage services on multiple hosts. Just mount the network interface, cpu, ram and start multiple processes...
      - yjftsjthsd-h 6y ago
        Nah, we'd still find ways to make it complex. I mean, yes, I think Plan 9 would've been a nicer base to work from and maybe reduced it a bit, but people are excellent at making things more complex, sometimes for legitimate reasons. Off the top of my head, I don't think Plan 9 has a good "orchestration" story itself (ex. "run 17 instances of this process wherever there's enough free resources in our 100-server cluster"), and while the "Plan 9 k8s" of this alternate universe would be simpler than the k8s we have in reality, it would still exist and add one or more layers of abstraction, and then people would stick management layers in front of that, and...
        ColanR 6y ago
        > I don't think Plan 9 has a good "orchestration" story itself I'm not super familiar with plan 9, but given what I do know, I'm pretty sure that could be done with an rc script. I'm pretty sure that a) you can fork processes after starting them on remote systems, and b) you can query remote systems to figure out which ones have free resources. Given that, it's just a shell / rc script away. Then you just need another script to do the management. ;)
        nitrogen 6y ago
        As another example of something orchestration-like ("run 17 processes where possible on this 100 node cluster"), there were also OpenMOSIX and Beowulf clusters on Linux, which if I remember right would load balance work across available systems, in the same way that the kernel's scheduler balances work across CPU cores. https://en.wikipedia.org/wiki/OpenMosix https://en.wikipedia.org/wiki/OpenMosix
- b2ccb2 6y ago
  Nitpick... LXC containers is the equivalent to saying the HIV virus, or more recently SARS-CoV-2 virus
  - jlgaddis 6y ago
    Nobody cares.
kbenson 6y ago
> If you're running Kubernetes, you're almost always still running Docker. Red Hat is trying to base all their stuff on Podman, which is Docker rewritten to by a script instead of a client/server (but can be renamed and used as a drop-in Docker replacement with the same command API). If they have their way, that statement will not longer be quite as correct.
- Spivak 6y ago
  Podman is meant as a replacement for the docker command as it’s typically used by developers and for bespoke services that can use systemd for lifecycle management. podman is an ergonomic runc wrapper. Redhat’s answer to capital D Docker is cri-o.
  - CameronNemo 6y ago
    To add to this, cri-o and libpod/podman are increasingly sharing code.
antonvs 6y ago
> If you're running Kubernetes, you're almost always still running Docker. I'm not sure that's true - a lot of Kubernetes system use containerd these days.
- jsmith45 6y ago
  I think a lot of documentation still tends to suggest docker as a default, but containerd actually makes more sense. Why use something higher level than you need that wraps containerd (namely Docker) instead of something that is exactly at the level you want: containerd itself? The only real advantage of wrapping docker is that doing so ensures the docker cli works for introspecting containers. The Docker folks don't want people to be wrapping dockerd, because then those infrastructure folk complain about changes/additional features in dockerd. Even dockerd is (glacially) moving in the direction of move heavily using containerd, for example using it to pull/store images. The longer-term intent appears to be for dockerd being more for implementing docker specific stuff like its networking, swarm/stack, and various features needed by the front-end.

WestCoastJustin 6y ago

Yeah, funny how it all worked out. cgroups/namespaces powered simple LXC containers. Docker comes along and makes a nice workflow and package management system (wrapping cgroups/namespaces). Kubernetes comes along and makes a nice workflow/cluster management layer (wrapping Docker). Cloud providers come along and make a nice Kubernetes management layer (wrapping Kubernetes). Pretty crazy to see the evolution over the past few years. This tech has completely changed the sysadmin landscape/job descriptions and sort of threw tons of gas the whole devops movement. Disclaimer: I worked at both Docker & Google. Although not on this tech specifically. Opinions are my own here.

tyingq 6y ago
"threw tons of gas on the whole devops movement" I'm happy to find that I agree with you on at least one area of this new ecosystem :)
- 0xbadcafebee 6y ago
  This is literally my job and I kind of hate how unnecessarily complex it all is. Not in the sense of "distributed systems are hard", but that we don't actually need the onion to have 20 layers, we really only need 2 or 3. The reason we need an orchestration system for our orchestration system, is the orchestration system is a snowflake. We need an orchestration system because the container system is a snowflake. We need a container system because containers are a snowflake. None of it is really standard or easy to implement, because none of it was designed in the spirit of Unix. It was all just some random companies who threw some crap together so they could start making money selling ads or hosting services. (I don't mean cgroups, I mean the tools that use them, although cgroups are kind of warty in themselves) All most people need to do is run a couple processes on a couple hosts, send packets between services (and networks), and have something start/stop/restart those processes. You can have all of that with, say, systemd (ugh), or three different standard interfaces that any program can use with standard libraries/ABIs. Notice I didn't say "with 5 different daemons running on 3 hosts that need complex configuration, constant maintenance, and a migration effort every 3 months". The Open Container Initiative seems close to getting the first thing done. If we get the other two standardized, we can lose a whole bunch of the onion layers. Consequently a lot of people will need to find a new way to make money, because there'll be not nearly as much need to pay people to deal with the onion.
  - baq 6y ago
    The way you described it one has to wonder what would the tech landscape look like had Plan 9 received adoption instead of Linux, so you didn’t have to invent all those layers of complexity to manage services on multiple hosts. Just mount the network interface, cpu, ram and start multiple processes...
    - yjftsjthsd-h 6y ago
      Nah, we'd still find ways to make it complex. I mean, yes, I think Plan 9 would've been a nicer base to work from and maybe reduced it a bit, but people are excellent at making things more complex, sometimes for legitimate reasons. Off the top of my head, I don't think Plan 9 has a good "orchestration" story itself (ex. "run 17 instances of this process wherever there's enough free resources in our 100-server cluster"), and while the "Plan 9 k8s" of this alternate universe would be simpler than the k8s we have in reality, it would still exist and add one or more layers of abstraction, and then people would stick management layers in front of that, and...
      - ColanR 6y ago
        > I don't think Plan 9 has a good "orchestration" story itself I'm not super familiar with plan 9, but given what I do know, I'm pretty sure that could be done with an rc script. I'm pretty sure that a) you can fork processes after starting them on remote systems, and b) you can query remote systems to figure out which ones have free resources. Given that, it's just a shell / rc script away. Then you just need another script to do the management. ;)
        nitrogen 6y ago
        As another example of something orchestration-like ("run 17 processes where possible on this 100 node cluster"), there were also OpenMOSIX and Beowulf clusters on Linux, which if I remember right would load balance work across available systems, in the same way that the kernel's scheduler balances work across CPU cores. https://en.wikipedia.org/wiki/OpenMosix https://en.wikipedia.org/wiki/OpenMosix
b2ccb2 6y ago
Nitpick... LXC containers is the equivalent to saying the HIV virus, or more recently SARS-CoV-2 virus
- jlgaddis 6y ago
  Nobody cares.

kbenson 6y ago

> If you're running Kubernetes, you're almost always still running Docker. Red Hat is trying to base all their stuff on Podman, which is Docker rewritten to by a script instead of a client/server (but can be renamed and used as a drop-in Docker replacement with the same command API). If they have their way, that statement will not longer be quite as correct.

Spivak 6y ago
Podman is meant as a replacement for the docker command as it’s typically used by developers and for bespoke services that can use systemd for lifecycle management. podman is an ergonomic runc wrapper. Redhat’s answer to capital D Docker is cri-o.
- CameronNemo 6y ago
  To add to this, cri-o and libpod/podman are increasingly sharing code.

antonvs 6y ago

> If you're running Kubernetes, you're almost always still running Docker. I'm not sure that's true - a lot of Kubernetes system use containerd these days.

jsmith45 6y ago
I think a lot of documentation still tends to suggest docker as a default, but containerd actually makes more sense. Why use something higher level than you need that wraps containerd (namely Docker) instead of something that is exactly at the level you want: containerd itself? The only real advantage of wrapping docker is that doing so ensures the docker cli works for introspecting containers. The Docker folks don't want people to be wrapping dockerd, because then those infrastructure folk complain about changes/additional features in dockerd. Even dockerd is (glacially) moving in the direction of move heavily using containerd, for example using it to pull/store images. The longer-term intent appears to be for dockerd being more for implementing docker specific stuff like its networking, swarm/stack, and various features needed by the front-end.