4 ms·

Yeah, funny how it all worked out. cgroups/namespaces powered simple LXC containers. Docker comes along and makes a nice workflow and package management system

Yeah, funny how it all worked out. cgroups/namespaces powered simple LXC containers. Docker comes along and makes a nice workflow and package management system (wrapping cgroups/namespaces). Kubernetes comes along and makes a nice workflow/cluster management layer (wrapping Docker). Cloud providers come along and make a nice Kubernetes management layer (wrapping Kubernetes). Pretty crazy to see the evolution over the past few years. This tech has completely changed the sysadmin landscape/job descriptions and sort of threw tons of gas the whole devops movement. Disclaimer: I worked at both Docker & Google. Although not on this tech specifically. Opinions are my own here.

tyingq 6y ago
"threw tons of gas on the whole devops movement" I'm happy to find that I agree with you on at least one area of this new ecosystem :)
- 0xbadcafebee 6y ago
  This is literally my job and I kind of hate how unnecessarily complex it all is. Not in the sense of "distributed systems are hard", but that we don't actually need the onion to have 20 layers, we really only need 2 or 3. The reason we need an orchestration system for our orchestration system, is the orchestration system is a snowflake. We need an orchestration system because the container system is a snowflake. We need a container system because containers are a snowflake. None of it is really standard or easy to implement, because none of it was designed in the spirit of Unix. It was all just some random companies who threw some crap together so they could start making money selling ads or hosting services. (I don't mean cgroups, I mean the tools that use them, although cgroups are kind of warty in themselves) All most people need to do is run a couple processes on a couple hosts, send packets between services (and networks), and have something start/stop/restart those processes. You can have all of that with, say, systemd (ugh), or three different standard interfaces that any program can use with standard libraries/ABIs. Notice I didn't say "with 5 different daemons running on 3 hosts that need complex configuration, constant maintenance, and a migration effort every 3 months". The Open Container Initiative seems close to getting the first thing done. If we get the other two standardized, we can lose a whole bunch of the onion layers. Consequently a lot of people will need to find a new way to make money, because there'll be not nearly as much need to pay people to deal with the onion.
  - baq 6y ago
    The way you described it one has to wonder what would the tech landscape look like had Plan 9 received adoption instead of Linux, so you didn’t have to invent all those layers of complexity to manage services on multiple hosts. Just mount the network interface, cpu, ram and start multiple processes...
    - yjftsjthsd-h 6y ago
      Nah, we'd still find ways to make it complex. I mean, yes, I think Plan 9 would've been a nicer base to work from and maybe reduced it a bit, but people are excellent at making things more complex, sometimes for legitimate reasons. Off the top of my head, I don't think Plan 9 has a good "orchestration" story itself (ex. "run 17 instances of this process wherever there's enough free resources in our 100-server cluster"), and while the "Plan 9 k8s" of this alternate universe would be simpler than the k8s we have in reality, it would still exist and add one or more layers of abstraction, and then people would stick management layers in front of that, and...
      - ColanR 6y ago
        > I don't think Plan 9 has a good "orchestration" story itself I'm not super familiar with plan 9, but given what I do know, I'm pretty sure that could be done with an rc script. I'm pretty sure that a) you can fork processes after starting them on remote systems, and b) you can query remote systems to figure out which ones have free resources. Given that, it's just a shell / rc script away. Then you just need another script to do the management. ;)
        nitrogen 6y ago
        As another example of something orchestration-like ("run 17 processes where possible on this 100 node cluster"), there were also OpenMOSIX and Beowulf clusters on Linux, which if I remember right would load balance work across available systems, in the same way that the kernel's scheduler balances work across CPU cores. https://en.wikipedia.org/wiki/OpenMosix https://en.wikipedia.org/wiki/OpenMosix
b2ccb2 6y ago
Nitpick... LXC containers is the equivalent to saying the HIV virus, or more recently SARS-CoV-2 virus
- jlgaddis 6y ago
  Nobody cares.

tyingq 6y ago

"threw tons of gas on the whole devops movement" I'm happy to find that I agree with you on at least one area of this new ecosystem :)

0xbadcafebee 6y ago
This is literally my job and I kind of hate how unnecessarily complex it all is. Not in the sense of "distributed systems are hard", but that we don't actually need the onion to have 20 layers, we really only need 2 or 3. The reason we need an orchestration system for our orchestration system, is the orchestration system is a snowflake. We need an orchestration system because the container system is a snowflake. We need a container system because containers are a snowflake. None of it is really standard or easy to implement, because none of it was designed in the spirit of Unix. It was all just some random companies who threw some crap together so they could start making money selling ads or hosting services. (I don't mean cgroups, I mean the tools that use them, although cgroups are kind of warty in themselves) All most people need to do is run a couple processes on a couple hosts, send packets between services (and networks), and have something start/stop/restart those processes. You can have all of that with, say, systemd (ugh), or three different standard interfaces that any program can use with standard libraries/ABIs. Notice I didn't say "with 5 different daemons running on 3 hosts that need complex configuration, constant maintenance, and a migration effort every 3 months". The Open Container Initiative seems close to getting the first thing done. If we get the other two standardized, we can lose a whole bunch of the onion layers. Consequently a lot of people will need to find a new way to make money, because there'll be not nearly as much need to pay people to deal with the onion.
- baq 6y ago
  The way you described it one has to wonder what would the tech landscape look like had Plan 9 received adoption instead of Linux, so you didn’t have to invent all those layers of complexity to manage services on multiple hosts. Just mount the network interface, cpu, ram and start multiple processes...
  - yjftsjthsd-h 6y ago
    Nah, we'd still find ways to make it complex. I mean, yes, I think Plan 9 would've been a nicer base to work from and maybe reduced it a bit, but people are excellent at making things more complex, sometimes for legitimate reasons. Off the top of my head, I don't think Plan 9 has a good "orchestration" story itself (ex. "run 17 instances of this process wherever there's enough free resources in our 100-server cluster"), and while the "Plan 9 k8s" of this alternate universe would be simpler than the k8s we have in reality, it would still exist and add one or more layers of abstraction, and then people would stick management layers in front of that, and...
    - ColanR 6y ago
      > I don't think Plan 9 has a good "orchestration" story itself I'm not super familiar with plan 9, but given what I do know, I'm pretty sure that could be done with an rc script. I'm pretty sure that a) you can fork processes after starting them on remote systems, and b) you can query remote systems to figure out which ones have free resources. Given that, it's just a shell / rc script away. Then you just need another script to do the management. ;)
      - nitrogen 6y ago
        As another example of something orchestration-like ("run 17 processes where possible on this 100 node cluster"), there were also OpenMOSIX and Beowulf clusters on Linux, which if I remember right would load balance work across available systems, in the same way that the kernel's scheduler balances work across CPU cores. https://en.wikipedia.org/wiki/OpenMosix https://en.wikipedia.org/wiki/OpenMosix

b2ccb2 6y ago

Nitpick... LXC containers is the equivalent to saying the HIV virus, or more recently SARS-CoV-2 virus

jlgaddis 6y ago
Nobody cares.