4 ms·

(Disclaimer; I wrote the blog post a little while ago, and my thoughts on this have changed a little bit - surprised to see it pop up here...) I think the conf

(Disclaimer; I wrote the blog post a little while ago, and my thoughts on this have changed a little bit - surprised to see it pop up here...) I think the confusion here is two-fold; sure, I think many don't really grasp what the point of the project is in its entirety, but I also think the aim of the project is a bit confused. The comment I made that you're referring to was specifically about the runtime: that's the bit of the paragraph at the start you chopped out. Yes, the runtime could change in the future, but the comment was about the existing runtime. The 'container standard' thing I think is potentially interesting, but actually, I don't think it buys much. As a set of tools, it is substantially weaker than the existing system development/spinning tools. And sure, getting rid of overlays might be possible - but then, what's the point? If you're going to flatten out trees, you may as well build the image properly in the first place.

derefr 13y ago
The point of the layered containers has nothing to do with how they're run; it has to do with how they're developed. By splitting the OS from the runtime from the application, each one can be updated when the layer above it needs it to be, and is otherwise fixed to a binary-identical image. Then, new releases of higher-level things (apps) can target the old versions of lower-level things (runtimes, OSes), knowing they will be literally, bit-for-bit, the same thing their other releases are using. This is the guarantee Heroku makes, for example: updates to their servers will never break your app, because although the packages making up their infrastructure might update, the packages making up your container's base image are frozen in time unless you switch out your container's "stack" (base image) for a new one. Having a frozen base OS image, and then a frozen runtime on top of that, allows for perfect repeatability in your deploy process. Once you've got a tested-and-working runtime image, that references a tested-and-working OS image, you just stop touching them altogether; you keep the container-ID of that runtime image fixed in production, and deploy your new app releases on top of it. One neat side-effect of this: if containers have parent-images in common, those parent-images can be cached at the target. If all the containers running on some server use the same common base-image, that base image only needs to be downloaded to the server once. The second-through-Nth time, the container only grabs what's different--the tiny "app" part of the image--and then composes it with everything into a running container.
- ealexhudson 13y ago
  I disagree entirely, tbh, the only advantage of layering I see is precisely in the runtime: it makes the layers that change most very slim and easily deployable. Repeatability is great, but who doesn't have that already? Are people really building new OS images for every deploy? I don't believe that for a second, and I can't think of many tools off the top of my head that don't have that baked in right from the start. That's the whole point of package management.
  - derefr 13y ago
    Package management is entirely non-repeatable in almost every incarnation. DEB, RPM, even Slackware tarballs, all run scripts on install. There are shunts of existing files and directories, interactive prompts, EULAs to agree to (try installing mscorefonts on Ubuntu), packages that immediately start services that generate files on first run that the service expects to be there from then on, packages that expect other packages to not have been installed (samba3 doesn't like samba4 much)--and so forth. Installing a package can fail, or produce widely-varying results from machine to machine. "Configuration management", ala Chef/Puppet, is no better--it tries to manipulate a system from an unknown state into a known state, without having an awareness of the full set of interactions that the unknown state can have on the known state. (For example, deploy Apache with default setup via Chef, on a server that already had Nginx installed manually. What do you get? Who knows!) You'd think that, say, running the OS install media from scratch on each "container-up", and then running a script to install a preset list of packages with hardcoded versions, might be enough--but nope, OS install media is absolutely non-deterministic, just like everything else. The installer could decide from the container's IP address that it should talk to a different package server (oh hey I'm in Canada let's use ca.archive.ubuntu.com!) and then find itself unable to get past the deploy-infrastructure's firewall-whitelist. In short--anything that relies upon, or can make a decision based upon, information that could be different from container to container (like the container's IP address, for example) isn't guaranteed to produce binary-identical results at the target. You only get that by running through whatever imperative process spits out all these files once--and then freezing the results into a declarative container. So, what does work? Xen snapshots. Amazon AMIs. Vagrant images. All of these are declarative. And all of these are target formats for Docker. Docker is a vendor-neutral thing which you will turn into these, along with turning it into its current LXC+aufs form. And note, by design, the running images of the "final product" will be leaf-nodes; you won't touch them or modify them or SSH into them, you won't base new containers on them; you'll just spin them up and then down again, like on-demand EC2 instances. Docker is not for doing fancy things with the running final products in their target format. Docker, by itself, is not for production at all! Once you've deployed a Docker container as a Xen snapshot or an AMI or whatever, it's done; some other infrastructure takes care of the running the target-format containers part. So what's all that junk in the Docker toolchain? Docker, as a standard, is an intermediary format that makes it easy for developers to build these vendor-neutral images. The reason you can start containers, stop them, freeze them, and then fork new containers from them, is entirely to do with developing new container images, and not at all to do with deploying containers in production. It's about re-using common parent images, by reference, as easily as we currently stick a Github repo in our Gemfiles.
nickstinemates 13y ago
Take a look at the latest blog post, which should address the confusion you mention: http://blog.docker.io/2013/08/getting-to-docker-1-0/ http://blog.docker.io/2013/08/getting-to-docker-1-0/. The blog addresses the direction that Docker is headed in and who the core audience of the tool should be.

derefr 13y ago

The point of the layered containers has nothing to do with how they're run; it has to do with how they're developed. By splitting the OS from the runtime from the application, each one can be updated when the layer above it needs it to be, and is otherwise fixed to a binary-identical image. Then, new releases of higher-level things (apps) can target the old versions of lower-level things (runtimes, OSes), knowing they will be literally, bit-for-bit, the same thing their other releases are using. This is the guarantee Heroku makes, for example: updates to their servers will never break your app, because although the packages making up their infrastructure might update, the packages making up your container's base image are frozen in time unless you switch out your container's "stack" (base image) for a new one. Having a frozen base OS image, and then a frozen runtime on top of that, allows for perfect repeatability in your deploy process. Once you've got a tested-and-working runtime image, that references a tested-and-working OS image, you just stop touching them altogether; you keep the container-ID of that runtime image fixed in production, and deploy your new app releases on top of it. One neat side-effect of this: if containers have parent-images in common, those parent-images can be cached at the target. If all the containers running on some server use the same common base-image, that base image only needs to be downloaded to the server once. The second-through-Nth time, the container only grabs what's different--the tiny "app" part of the image--and then composes it with everything into a running container.

ealexhudson 13y ago
I disagree entirely, tbh, the only advantage of layering I see is precisely in the runtime: it makes the layers that change most very slim and easily deployable. Repeatability is great, but who doesn't have that already? Are people really building new OS images for every deploy? I don't believe that for a second, and I can't think of many tools off the top of my head that don't have that baked in right from the start. That's the whole point of package management.
- derefr 13y ago
  Package management is entirely non-repeatable in almost every incarnation. DEB, RPM, even Slackware tarballs, all run scripts on install. There are shunts of existing files and directories, interactive prompts, EULAs to agree to (try installing mscorefonts on Ubuntu), packages that immediately start services that generate files on first run that the service expects to be there from then on, packages that expect other packages to not have been installed (samba3 doesn't like samba4 much)--and so forth. Installing a package can fail, or produce widely-varying results from machine to machine. "Configuration management", ala Chef/Puppet, is no better--it tries to manipulate a system from an unknown state into a known state, without having an awareness of the full set of interactions that the unknown state can have on the known state. (For example, deploy Apache with default setup via Chef, on a server that already had Nginx installed manually. What do you get? Who knows!) You'd think that, say, running the OS install media from scratch on each "container-up", and then running a script to install a preset list of packages with hardcoded versions, might be enough--but nope, OS install media is absolutely non-deterministic, just like everything else. The installer could decide from the container's IP address that it should talk to a different package server (oh hey I'm in Canada let's use ca.archive.ubuntu.com!) and then find itself unable to get past the deploy-infrastructure's firewall-whitelist. In short--anything that relies upon, or can make a decision based upon, information that could be different from container to container (like the container's IP address, for example) isn't guaranteed to produce binary-identical results at the target. You only get that by running through whatever imperative process spits out all these files once--and then freezing the results into a declarative container. So, what does work? Xen snapshots. Amazon AMIs. Vagrant images. All of these are declarative. And all of these are target formats for Docker. Docker is a vendor-neutral thing which you will turn into these, along with turning it into its current LXC+aufs form. And note, by design, the running images of the "final product" will be leaf-nodes; you won't touch them or modify them or SSH into them, you won't base new containers on them; you'll just spin them up and then down again, like on-demand EC2 instances. Docker is not for doing fancy things with the running final products in their target format. Docker, by itself, is not for production at all! Once you've deployed a Docker container as a Xen snapshot or an AMI or whatever, it's done; some other infrastructure takes care of the running the target-format containers part. So what's all that junk in the Docker toolchain? Docker, as a standard, is an intermediary format that makes it easy for developers to build these vendor-neutral images. The reason you can start containers, stop them, freeze them, and then fork new containers from them, is entirely to do with developing new container images, and not at all to do with deploying containers in production. It's about re-using common parent images, by reference, as easily as we currently stick a Github repo in our Gemfiles.

nickstinemates 13y ago

Take a look at the latest blog post, which should address the confusion you mention: http://blog.docker.io/2013/08/getting-to-docker-1-0/ http://blog.docker.io/2013/08/getting-to-docker-1-0/. The blog addresses the direction that Docker is headed in and who the core audience of the tool should be.