3 ms·

I disagree entirely, tbh, the only advantage of layering I see is precisely in the runtime: it makes the layers that change most very slim and easily deployable

I disagree entirely, tbh, the only advantage of layering I see is precisely in the runtime: it makes the layers that change most very slim and easily deployable. Repeatability is great, but who doesn't have that already? Are people really building new OS images for every deploy? I don't believe that for a second, and I can't think of many tools off the top of my head that don't have that baked in right from the start. That's the whole point of package management.

derefr 13y ago
Package management is entirely non-repeatable in almost every incarnation. DEB, RPM, even Slackware tarballs, all run scripts on install. There are shunts of existing files and directories, interactive prompts, EULAs to agree to (try installing mscorefonts on Ubuntu), packages that immediately start services that generate files on first run that the service expects to be there from then on, packages that expect other packages to not have been installed (samba3 doesn't like samba4 much)--and so forth. Installing a package can fail, or produce widely-varying results from machine to machine. "Configuration management", ala Chef/Puppet, is no better--it tries to manipulate a system from an unknown state into a known state, without having an awareness of the full set of interactions that the unknown state can have on the known state. (For example, deploy Apache with default setup via Chef, on a server that already had Nginx installed manually. What do you get? Who knows!) You'd think that, say, running the OS install media from scratch on each "container-up", and then running a script to install a preset list of packages with hardcoded versions, might be enough--but nope, OS install media is absolutely non-deterministic, just like everything else. The installer could decide from the container's IP address that it should talk to a different package server (oh hey I'm in Canada let's use ca.archive.ubuntu.com!) and then find itself unable to get past the deploy-infrastructure's firewall-whitelist. In short--anything that relies upon, or can make a decision based upon, information that could be different from container to container (like the container's IP address, for example) isn't guaranteed to produce binary-identical results at the target. You only get that by running through whatever imperative process spits out all these files once--and then freezing the results into a declarative container. So, what does work? Xen snapshots. Amazon AMIs. Vagrant images. All of these are declarative. And all of these are target formats for Docker. Docker is a vendor-neutral thing which you will turn into these, along with turning it into its current LXC+aufs form. And note, by design, the running images of the "final product" will be leaf-nodes; you won't touch them or modify them or SSH into them, you won't base new containers on them; you'll just spin them up and then down again, like on-demand EC2 instances. Docker is not for doing fancy things with the running final products in their target format. Docker, by itself, is not for production at all! Once you've deployed a Docker container as a Xen snapshot or an AMI or whatever, it's done; some other infrastructure takes care of the running the target-format containers part. So what's all that junk in the Docker toolchain? Docker, as a standard, is an intermediary format that makes it easy for developers to build these vendor-neutral images. The reason you can start containers, stop them, freeze them, and then fork new containers from them, is entirely to do with developing new container images, and not at all to do with deploying containers in production. It's about re-using common parent images, by reference, as easily as we currently stick a Github repo in our Gemfiles.
- deleted 13y ago
  [deleted]
- emilisto 13y ago
  > Docker is not for doing fancy things with the running final products in their target format. Docker, by itself, is not for production at all! Once you've deployed a Docker container as a Xen snapshot or an AMI or whatever, it's done; some other infrastructure takes care of the running the target-format containers part. This is something very interesting and under-communicated. I've always assumed docker is run on the production server and is used to pull updated images and spawn containers. But you're suggesting one uses custom toolchain for making an image out of the container filesystem and the LXC template, and then deploy this container?
  - shykes 13y ago
    Both are possible. And since docker itself is not yet production-ready, exporting docker containers to "inert" server images (for example AMIs) is a good stopgap which allows you to use docker for dev and test. But that is not the ideal workflow. The ideal workflow is to run docker on all your machines, from development to production, and move containers across them. If you don't do that, you will miss out on a big part of the value of docker. To name a few: identical images in dev and prod; lightweight deployment; and a toolchain that is less dependent on a particular infrastructure.
- nickknw 13y ago
  This exchange has really helped me understand more about Docker and why and when to use it. Thanks!
vidarh 13y ago
> Repeatability is great, but who doesn't have that already? In my experience: Pretty much nobody does fully repeatable builds. Most people have a hodgepodge of "mostly similar" OS images that sometimes get updated in production, possibly in contradictory ways. You'll find the odd exception, but outside of large-ish companies they are few and far between, largely because while it certainly can be done, the toolchains people work with are cumbersome enough that for small teams it gets terribly tempting to e.g. "apt-get install" something on a live box rather than update their images and deploy. > That's the whole point of package management. Only if you disable it in production and only ever use the package management tools to update OS images.
shykes 13y ago
"Repeatability is great, but who doesn't have that already?" Almost everyone. I think this is the heart of the misunderstanding right there. 99% of developers don't have access to a repeatable deployment system. When they do, it severy limits what they can deploy, and where. You mention in your article that the "secret sauce" of docker is "not that clever". The point is not to be clever but to solve a problem. The fundamental technology for solving repeatable deployment has been available for years, and what do those in the know have to show for it? We're all still reinventing the wheel for every single damn project, wasting huge amounts of energy in the process.

derefr 13y ago

Package management is entirely non-repeatable in almost every incarnation. DEB, RPM, even Slackware tarballs, all run scripts on install. There are shunts of existing files and directories, interactive prompts, EULAs to agree to (try installing mscorefonts on Ubuntu), packages that immediately start services that generate files on first run that the service expects to be there from then on, packages that expect other packages to not have been installed (samba3 doesn't like samba4 much)--and so forth. Installing a package can fail, or produce widely-varying results from machine to machine. "Configuration management", ala Chef/Puppet, is no better--it tries to manipulate a system from an unknown state into a known state, without having an awareness of the full set of interactions that the unknown state can have on the known state. (For example, deploy Apache with default setup via Chef, on a server that already had Nginx installed manually. What do you get? Who knows!) You'd think that, say, running the OS install media from scratch on each "container-up", and then running a script to install a preset list of packages with hardcoded versions, might be enough--but nope, OS install media is absolutely non-deterministic, just like everything else. The installer could decide from the container's IP address that it should talk to a different package server (oh hey I'm in Canada let's use ca.archive.ubuntu.com!) and then find itself unable to get past the deploy-infrastructure's firewall-whitelist. In short--anything that relies upon, or can make a decision based upon, information that could be different from container to container (like the container's IP address, for example) isn't guaranteed to produce binary-identical results at the target. You only get that by running through whatever imperative process spits out all these files once--and then freezing the results into a declarative container. So, what does work? Xen snapshots. Amazon AMIs. Vagrant images. All of these are declarative. And all of these are target formats for Docker. Docker is a vendor-neutral thing which you will turn into these, along with turning it into its current LXC+aufs form. And note, by design, the running images of the "final product" will be leaf-nodes; you won't touch them or modify them or SSH into them, you won't base new containers on them; you'll just spin them up and then down again, like on-demand EC2 instances. Docker is not for doing fancy things with the running final products in their target format. Docker, by itself, is not for production at all! Once you've deployed a Docker container as a Xen snapshot or an AMI or whatever, it's done; some other infrastructure takes care of the running the target-format containers part. So what's all that junk in the Docker toolchain? Docker, as a standard, is an intermediary format that makes it easy for developers to build these vendor-neutral images. The reason you can start containers, stop them, freeze them, and then fork new containers from them, is entirely to do with developing new container images, and not at all to do with deploying containers in production. It's about re-using common parent images, by reference, as easily as we currently stick a Github repo in our Gemfiles.

deleted 13y ago
[deleted]
emilisto 13y ago
> Docker is not for doing fancy things with the running final products in their target format. Docker, by itself, is not for production at all! Once you've deployed a Docker container as a Xen snapshot or an AMI or whatever, it's done; some other infrastructure takes care of the running the target-format containers part. This is something very interesting and under-communicated. I've always assumed docker is run on the production server and is used to pull updated images and spawn containers. But you're suggesting one uses custom toolchain for making an image out of the container filesystem and the LXC template, and then deploy this container?
- shykes 13y ago
  Both are possible. And since docker itself is not yet production-ready, exporting docker containers to "inert" server images (for example AMIs) is a good stopgap which allows you to use docker for dev and test. But that is not the ideal workflow. The ideal workflow is to run docker on all your machines, from development to production, and move containers across them. If you don't do that, you will miss out on a big part of the value of docker. To name a few: identical images in dev and prod; lightweight deployment; and a toolchain that is less dependent on a particular infrastructure.
nickknw 13y ago
This exchange has really helped me understand more about Docker and why and when to use it. Thanks!

vidarh 13y ago

> Repeatability is great, but who doesn't have that already? In my experience: Pretty much nobody does fully repeatable builds. Most people have a hodgepodge of "mostly similar" OS images that sometimes get updated in production, possibly in contradictory ways. You'll find the odd exception, but outside of large-ish companies they are few and far between, largely because while it certainly can be done, the toolchains people work with are cumbersome enough that for small teams it gets terribly tempting to e.g. "apt-get install" something on a live box rather than update their images and deploy. > That's the whole point of package management. Only if you disable it in production and only ever use the package management tools to update OS images.

shykes 13y ago

"Repeatability is great, but who doesn't have that already?" Almost everyone. I think this is the heart of the misunderstanding right there. 99% of developers don't have access to a repeatable deployment system. When they do, it severy limits what they can deploy, and where. You mention in your article that the "secret sauce" of docker is "not that clever". The point is not to be clever but to solve a problem. The fundamental technology for solving repeatable deployment has been available for years, and what do those in the know have to show for it? We're all still reinventing the wheel for every single damn project, wasting huge amounts of energy in the process.