4 ms·

What exactly does containerizing everything give you anyways? I really don't get it. Before - use chef/puppet to manage dependencies, distribute config files.

What exactly does containerizing everything give you anyways? I really don't get it. Before - use chef/puppet to manage dependencies, distribute config files. run processes. maybe use something like upstart to restart on failure. After - use Dockerfiles to manage dependencies (same thing, bunch of install commands). now you have a container for the web app, one for another service, etc. so everything is isolated.. great. what do you gain over running 2 separate processes? thats pretty damn isolated too.. except for the same disk, they each have own virtual memory, state, config files etc.. I'm not a (modern) ops expert at all, but i know my way around the command line. What do you gain from Docker or say launching a mongo instance on the cloud instead of just renting a server and launching the process? I really want to know. Atleast on a small scale, say if you're managing say 10-20 servers, i don't see the point.

Scarblac 11y ago
I see an instance as sort of a compiled binary of your app and the environment it runs in. You can install your app, all its dependencies, get all the config files correct, run tests on the created instance, and then you have a Docker instance and can start up any number of them in production. If any of those steps fails in the middle of some script, you haven't put any server in some half way there state. If a rollback is needed, you can switch to the previous Docker instance, and changed requirements won't trip you up. We used to do it with zipped chroot environments and some startup/shutdown scripts, Docker is more or less that. Of course you need to store data outside them, as otherwise you lose once you switch to a newer version of your instance, but that's easy enough. All that said, still not really a fan of it.
- TheIronYuppie 11y ago
  Disclaimer: I work at Google on Kubernetes I'd love to hear more - anything in particular that doesn't feel like a fit?
  - Scarblac 11y ago
    Nothing specific, they do what they do quite well. I have a generic feeling that the rabbit hole is getting too deep though. In our case we run large servers with virtualisation software (I think they run on Windows, I never touch this layer). Then we have virtual Linux servers running on them. They have package managers that are also a way to solve this sort of problem. We run Docker instances on those servers. They are Linux again, so have their own package manager. Then we run Python (with virtualenv or Buildout) or Node (with npm), that again have their own package managers that try to provide isolated environments. And of course they run bytecode in the Python or Node... And that ridiculous stack is used to run some relatively mundane web app that can't even mutate its data directly, and is used to send some set of Javascript and HTML and JSON to the user's browser. Which is where the app actually runs... I wish we had some Web framework as nice to use as Django but compiling to static binaries that are immediately sort of an equivalent to a 12 factors app, and a kernel that was made for running them directly. Or so. But this is the internet, right? It literally evolves, just like nature -- we can only build on top of existing layers, not remove a few and start over.
dpeterson 11y ago
Even on a small scale there are significant benefits, both in production and in development and test. As the article states, containers are great for moving away from a monolith architecture. Many applications are moving to a distributed service oriented architecture. Each piece can be deployed separate. That leads to an easier more maintainable development process. However, the larger number of independently deployable apps adds operation complexity. Trying to manage that complexity, even on a small scale is no fun. Queue containers. Containers by themselves are interesting; they let you run many more applications on a single machine. Unlike a VM, there is no operating system chewing up ram and cpu. On machines only able to run 4 VM's reasonably well, I have seen nearly a hundred containers. That is only a small part. Container platforms such as Openshift make the process much nicer and more powerful. Openshift uses Kubernetes under the covers. You arrange your applications inside pods. The pods are wrapped in a Kubernetes service. The pods automatically assign internal ip addresses; that allows you to put, let's say, 20 wildfly servers on the same machine without worrying about ip and port clashes. The service acts as a load balancer and a router directs traffic accordingly. I don't have to worry about any of that. It is all automated. I create a build config and a deployment config for my services (there could be hundreds) then Openshift (or other container PAAS) maximizes the potential of my hardware. In development, it is much easier to completely replicate production. Everything is encapsulated in the container image. Not just the application; all of the software and configuration the application requires is part of the image. If I need to recreate the container image, every step is in the Dockerfile. I do not need documentation to remind me how everything is set up. In openshift, I grab a template, then click a button; within seconds, the containers of my app are running. That would take much longer with VMs. Containers allow me to develop in a production like environment because I am able to deploy an unscaled version easily. What took multiple physical machines to replicate now takes a few seconds to deploy on a laptop.
cwp 11y ago
The point is better hardware utilization. Part of the "isolation" that containers provide is resource allocation. You assign limits to the CPU cycles, RAM, disk IO, network IO etc that each container can use. Without any kind of virtualization, i.e. processes in an OS running on bare metal, processes can starve each other of resources - hog the CPU, fill up the disk etc. Virtual machines like kvm and Xen limit the resource usage of guest operating systems, but that comes with a lot of overhead. Each VM has to run its own kernel and a bunch of user-space programs in addition to the the service you're trying to provide. The disk images have to include a full operating system as well - config files, libraries, executables and so on. With containers you get the best of both worlds: the resource limits of virtual machines with the low overhead of processes sharing an OS. That means you can run multiple services per machine without them interfering with one another, and you can pack more of them onto each machine than if you were using virtual machines. So if you're Google (which did the initial kernel work to support containers in Linux) you can get a lot more computation out of your 100K-server data centre if you package all your software as containers and write sophisticated software to distribute them to server and shuffle them around as load fluctuates and batch jobs get run. For those of us that don't run our own data centres the benefit is much less clear. Running containers on top of virtual machines seems like a pretty terrible idea. With all that overhead, performance suffers and you end up running more containers to compensate, which just makes everything more complicated and more expensive. I think the current container craze boils down to two things: first, no matter what ugly, undocumented, unspecified process you use to build a container image, once you have the image you can deploy it in a repeatable, consistent way. That's a big improvement over Chef/Puppet right there. Second, fashion. Doing ops the Google way is cool, and it'll make my resume look good even if I'm not currently in a position to reap the benefits that Google does. Being a Puppet expert is so passé.
- TheIronYuppie 11y ago
  Disclaimer: I work at Google on Kubernetes Portability, as you mentioned, is huge - but beyond that there is also binpacking benefits. You can basically cut your serving costs by 50% or more by using lots of single process containers instead of poorly utilized VMs.
TheIronYuppie 11y ago
Disclaimer: I work at Google on Kubernetes The number one feature we see people get out of containers (as we did at Google) is flexibility. Bin packing and (some) isolation is important and valuable, but the vast majority is just being able to throw 25k containers at a system (Borg for internal work, now Kubernetes externally) and just say "Go run this somewhere." The system figures it out for you - and you never have to deal with server config. Then if you need to move to a new data center - presto, happens in seconds.

Scarblac 11y ago

I see an instance as sort of a compiled binary of your app and the environment it runs in. You can install your app, all its dependencies, get all the config files correct, run tests on the created instance, and then you have a Docker instance and can start up any number of them in production. If any of those steps fails in the middle of some script, you haven't put any server in some half way there state. If a rollback is needed, you can switch to the previous Docker instance, and changed requirements won't trip you up. We used to do it with zipped chroot environments and some startup/shutdown scripts, Docker is more or less that. Of course you need to store data outside them, as otherwise you lose once you switch to a newer version of your instance, but that's easy enough. All that said, still not really a fan of it.

TheIronYuppie 11y ago
Disclaimer: I work at Google on Kubernetes I'd love to hear more - anything in particular that doesn't feel like a fit?
- Scarblac 11y ago
  Nothing specific, they do what they do quite well. I have a generic feeling that the rabbit hole is getting too deep though. In our case we run large servers with virtualisation software (I think they run on Windows, I never touch this layer). Then we have virtual Linux servers running on them. They have package managers that are also a way to solve this sort of problem. We run Docker instances on those servers. They are Linux again, so have their own package manager. Then we run Python (with virtualenv or Buildout) or Node (with npm), that again have their own package managers that try to provide isolated environments. And of course they run bytecode in the Python or Node... And that ridiculous stack is used to run some relatively mundane web app that can't even mutate its data directly, and is used to send some set of Javascript and HTML and JSON to the user's browser. Which is where the app actually runs... I wish we had some Web framework as nice to use as Django but compiling to static binaries that are immediately sort of an equivalent to a 12 factors app, and a kernel that was made for running them directly. Or so. But this is the internet, right? It literally evolves, just like nature -- we can only build on top of existing layers, not remove a few and start over.

dpeterson 11y ago

Even on a small scale there are significant benefits, both in production and in development and test. As the article states, containers are great for moving away from a monolith architecture. Many applications are moving to a distributed service oriented architecture. Each piece can be deployed separate. That leads to an easier more maintainable development process. However, the larger number of independently deployable apps adds operation complexity. Trying to manage that complexity, even on a small scale is no fun. Queue containers. Containers by themselves are interesting; they let you run many more applications on a single machine. Unlike a VM, there is no operating system chewing up ram and cpu. On machines only able to run 4 VM's reasonably well, I have seen nearly a hundred containers. That is only a small part. Container platforms such as Openshift make the process much nicer and more powerful. Openshift uses Kubernetes under the covers. You arrange your applications inside pods. The pods are wrapped in a Kubernetes service. The pods automatically assign internal ip addresses; that allows you to put, let's say, 20 wildfly servers on the same machine without worrying about ip and port clashes. The service acts as a load balancer and a router directs traffic accordingly. I don't have to worry about any of that. It is all automated. I create a build config and a deployment config for my services (there could be hundreds) then Openshift (or other container PAAS) maximizes the potential of my hardware. In development, it is much easier to completely replicate production. Everything is encapsulated in the container image. Not just the application; all of the software and configuration the application requires is part of the image. If I need to recreate the container image, every step is in the Dockerfile. I do not need documentation to remind me how everything is set up. In openshift, I grab a template, then click a button; within seconds, the containers of my app are running. That would take much longer with VMs. Containers allow me to develop in a production like environment because I am able to deploy an unscaled version easily. What took multiple physical machines to replicate now takes a few seconds to deploy on a laptop.

cwp 11y ago

The point is better hardware utilization. Part of the "isolation" that containers provide is resource allocation. You assign limits to the CPU cycles, RAM, disk IO, network IO etc that each container can use. Without any kind of virtualization, i.e. processes in an OS running on bare metal, processes can starve each other of resources - hog the CPU, fill up the disk etc. Virtual machines like kvm and Xen limit the resource usage of guest operating systems, but that comes with a lot of overhead. Each VM has to run its own kernel and a bunch of user-space programs in addition to the the service you're trying to provide. The disk images have to include a full operating system as well - config files, libraries, executables and so on. With containers you get the best of both worlds: the resource limits of virtual machines with the low overhead of processes sharing an OS. That means you can run multiple services per machine without them interfering with one another, and you can pack more of them onto each machine than if you were using virtual machines. So if you're Google (which did the initial kernel work to support containers in Linux) you can get a lot more computation out of your 100K-server data centre if you package all your software as containers and write sophisticated software to distribute them to server and shuffle them around as load fluctuates and batch jobs get run. For those of us that don't run our own data centres the benefit is much less clear. Running containers on top of virtual machines seems like a pretty terrible idea. With all that overhead, performance suffers and you end up running more containers to compensate, which just makes everything more complicated and more expensive. I think the current container craze boils down to two things: first, no matter what ugly, undocumented, unspecified process you use to build a container image, once you have the image you can deploy it in a repeatable, consistent way. That's a big improvement over Chef/Puppet right there. Second, fashion. Doing ops the Google way is cool, and it'll make my resume look good even if I'm not currently in a position to reap the benefits that Google does. Being a Puppet expert is so passé.

TheIronYuppie 11y ago
Disclaimer: I work at Google on Kubernetes Portability, as you mentioned, is huge - but beyond that there is also binpacking benefits. You can basically cut your serving costs by 50% or more by using lots of single process containers instead of poorly utilized VMs.

TheIronYuppie 11y ago

Disclaimer: I work at Google on Kubernetes The number one feature we see people get out of containers (as we did at Google) is flexibility. Bin packing and (some) isolation is important and valuable, but the vast majority is just being able to throw 25k containers at a system (Borg for internal work, now Kubernetes externally) and just say "Go run this somewhere." The system figures it out for you - and you never have to deal with server config. Then if you need to move to a new data center - presto, happens in seconds.