5 ms·

I was ready to come in swinging in defending Docker, but I found myself agreeing with most of the points after spending a lot of time with Docker over the last

I was ready to come in swinging in defending Docker, but I found myself agreeing with most of the points after spending a lot of time with Docker over the last month (ported my Rails application's provisioning from Ansible to it). I would add to the list that it is currently hard to even find decent images of popular services that you would trust deploying to production (e.g. Postgres). I see with the launch of Docker Hub that they have some flagged as "official" now, but for example the Postgres one is a black box (no Dockerfile source available - not a "trusted build") so I can't trust it.[1] I've had to spend time porting my Ansible playbooks over to Dockerfiles due to this. I think part of the problem is that composition of images is strict "subclass"-type inheritance, so they don't compose easily. So it's hard to maintain a "vanilla" image rather than a monolithic one that has everything you need to run your particular service - so people just keep their images to themselves. For example, if I want a Ruby image that has the latest nginx and git installed, I can't just blend three images together. I have to pick a base image, and then manually add the rest myself via Dockerfile. Also, although Vagrant 1.6 added Docker syntax, you really have to understand how Docker works in order to use it. If you're learning Docker I'd stick with the vanilla `docker` syntax first in building your images, maybe using something like fig[2] to start with. At the end of the day it's another abstraction to manage. It does bring great benefits in my opinion, but the learning curve and time investment aren't cheap, so if you already have a suite of provisioning scripts it may not be worth it to make the leap yet. [1]: https://registry.hub.docker.com/_/postgres/ https://registry.hub.docker.com/_/postgres/ [2]: http://orchardup.github.io/fig/ http://orchardup.github.io/fig/

mwcampbell 12y ago
I don't know why the official Docker images aren't marked as trusted builds, but you can find sources here: https://github.com/docker-library https://github.com/docker-library
liamzebedee 12y ago
> it is currently hard to even find decent images of popular services Very true. I've yet to find a suitable image for a simple Nginx + Unicorn + Postgres + Rails setup.
- pyre 12y ago
  I thought that the point of Docker was to encapsulate each of those things into their own containers?
  - sanityinc 12y ago
    Not necessarily. The article covers that point.
    - koffiezet 12y ago
      And that point is, in my opinion, stupid. I run docker in production here, and if I had to make a docker image for each application with all services integrated, that would become a maintenance nightmare the moment you have to manage more than say 10 images (!not containers) built like that. You lose a lot of the flexibility by integrating everything in one big pre-configured blackbox. In some cases that's nice, like when you actually want to ship a blackbox to customers, but when managing infrastructure? Thanks but no thanks. When you have proper configuration management in place, spinning up and configuring an extra postgres, apache, nginx, jboss, redis, ... whatever container should be dead-easy, otherwise you're doing something wrong.
shykes 12y ago
> I see with the launch of Docker Hub that they have some flagged as "official" now, but for example the Postgres one is a black box (no Dockerfile source available - not a "trusted build") so I can't trust it.[1] It is actually built from source - by a separate open-source project called "stackbrew" which is basically the "homebrew of official Docker images". It was setup by the Docker maintainers before Hub introduced auto-builds, and then it just stuck. The idea is to allow anyone to participate in the maintenance effort, contribute new images etc. by simply making a pull request (again, just like Homebrew). Here's the source URL from which that Postgres image is built: https://github.com/dotcloud/stackbrew/blob/master/library/postgres https://github.com/dotcloud/stackbrew/blob/master/library/po...
ollysb 12y ago
>> I think part of the problem is that composition of images is strict "subclass"-type inheritance, so they don't compose easily. This does seem like a pretty major limitation, is this a fundamental constraint or is it something that might be fixed in the future?
- zaroth 12y ago
  The "subclass"-type inheritance is due to the underlying union filesystem (AUFS). Each command in the Dockerfile is applied as a layer and then cached, and each resulting image can be referenced as a starting point from another Dockerfile. So you can branch out from a common base image, but you can't somehow co-mingle multiple parent images into one child. If you think about it like the filesystem giving you a snapshot at each line of the Dockerfile, you can see how inheriting from a single snapshot is trivial, but somehow inheriting from multiple snapshots into one would require much more awareness in order to merge them. The practical effect is that either you build a more one-size-fits-all "base" image which you reference for all your services, or you have multiple base images with different more tailored sets of dependencies built up, or you have a very thin base image (e.g. not much more than ubuntu-latest) and just add what you need inside each individual Dockerfile. I ran into the same issue at first when laying out the Dockerfile hierarchy for all my services/roles, but I think in this case it actually falls under "less magic is better" / KISS. Dockerfiles are so easy to read and write once you get going, you won't mind the simplistic approach.
  - ollysb 12y ago
    It sounds like you need something else to help you build the dockerfiles then i.e. I need a dockerfile with nginx, postgres and memcached so I use something like puppet/chef to put it together, is that possible?
    - nl 12y ago
      It's easy to build new images from Dockerfiles. And "cut-n-paste" based "inheritance" works well when combining two or more Dockerfiles into a single one that has everything you want. But you can't use composition to combine the actual downloaded images themselves. Using Chef within Docker to build up the services is documented on the Docker site[1], and of course puppet or whatever is essentially the same. [1] http://docs.docker.com/articles/chef/ http://docs.docker.com/articles/chef/
vidarh 12y ago
> For example, if I want a Ruby image that has the latest nginx and git installed, I can't just blend three images together. I have to pick a base image, and then manually add the rest myself via Dockerfile. My solution to this is to avoid combining stuff in the same image. E.g. my home server has an image for haproxy that splits incoming requests to my various test web apps by hostname. Each of those web apps run in their own docker container that bind-mount their appropriate directory from my home dir. Then the container(s) I do my development in likewise bind-mount the appropriate directory. For the most part I ssh in to a single screen session, but for anything that has particular dependencies, I have separate containers (e.g. I keep a Ruby 1.8 container available to test code that should retain compatibility). So far, all of my Docker containers are extremely small for this reason: I compose by orchestrating multiple Docker containers, not by stuffing multiple things into each container. This for the most part means that dependencies I need to manually add rarely is more than installing an extra package with apt, or installing another gem. I agree with you that it's hard to find decent images to rely on. But by keeping the containers small and single purpose I find that it's not that often it matters - the generic part of the config is often a matter of a couple of lines, while the Dockerfile and associated config files etc. tend to be dominated by my personal requirements.
peterwwillis 12y ago
> For example, if I want a Ruby image that has the latest nginx and git installed, I can't just blend three images together. I have to pick a base image, and then manually add the rest myself via Dockerfile. Wouldn't it be just crazy if someone maintained a list of software they built to be modularly installed along with other optional dependencies compiled on a common architecture and operating system for a common base image? We could make a tool to download them from a list of approved sources and verify that the files were cryptographically signed by a particular builder. We could also have the dependent images downloaded automatically. Then we could extend scripting support to run custom-built tools to tailor the install to what was on the system at the time of install or uninstall. Heck, considering all that they could make it cross-platform or even multi-platform. Maybe they could call it the Really Professional-docker-image Manager, or the Docker Professional Knowledge Gem, or the Yodeler's Ultimate Maintainer package. That would be crazy. Somebody should create a startup for this ASAP. [note to NSA crawlers: this has been an example of sarcasm]
asb 12y ago
It seems a shame that Ubuntu's Juju has been passed over in this space. Their charm store is designed to make it easy to mix and match recipes, have the community endorse and improve them. See the 'features' tab on https://jujucharms.com/sidebar/search/precise/postgresql-71/?text=postgresql#summary https://jujucharms.com/sidebar/search/precise/postgresql-71/... for a nice summary of what is offered by the most popular postgres charm.

mwcampbell 12y ago

I don't know why the official Docker images aren't marked as trusted builds, but you can find sources here: https://github.com/docker-library https://github.com/docker-library

liamzebedee 12y ago

> it is currently hard to even find decent images of popular services Very true. I've yet to find a suitable image for a simple Nginx + Unicorn + Postgres + Rails setup.

pyre 12y ago
I thought that the point of Docker was to encapsulate each of those things into their own containers?
- sanityinc 12y ago
  Not necessarily. The article covers that point.
  - koffiezet 12y ago
    And that point is, in my opinion, stupid. I run docker in production here, and if I had to make a docker image for each application with all services integrated, that would become a maintenance nightmare the moment you have to manage more than say 10 images (!not containers) built like that. You lose a lot of the flexibility by integrating everything in one big pre-configured blackbox. In some cases that's nice, like when you actually want to ship a blackbox to customers, but when managing infrastructure? Thanks but no thanks. When you have proper configuration management in place, spinning up and configuring an extra postgres, apache, nginx, jboss, redis, ... whatever container should be dead-easy, otherwise you're doing something wrong.

shykes 12y ago

> I see with the launch of Docker Hub that they have some flagged as "official" now, but for example the Postgres one is a black box (no Dockerfile source available - not a "trusted build") so I can't trust it.[1] It is actually built from source - by a separate open-source project called "stackbrew" which is basically the "homebrew of official Docker images". It was setup by the Docker maintainers before Hub introduced auto-builds, and then it just stuck. The idea is to allow anyone to participate in the maintenance effort, contribute new images etc. by simply making a pull request (again, just like Homebrew). Here's the source URL from which that Postgres image is built: https://github.com/dotcloud/stackbrew/blob/master/library/postgres https://github.com/dotcloud/stackbrew/blob/master/library/po...

ollysb 12y ago

>> I think part of the problem is that composition of images is strict "subclass"-type inheritance, so they don't compose easily. This does seem like a pretty major limitation, is this a fundamental constraint or is it something that might be fixed in the future?

zaroth 12y ago
The "subclass"-type inheritance is due to the underlying union filesystem (AUFS). Each command in the Dockerfile is applied as a layer and then cached, and each resulting image can be referenced as a starting point from another Dockerfile. So you can branch out from a common base image, but you can't somehow co-mingle multiple parent images into one child. If you think about it like the filesystem giving you a snapshot at each line of the Dockerfile, you can see how inheriting from a single snapshot is trivial, but somehow inheriting from multiple snapshots into one would require much more awareness in order to merge them. The practical effect is that either you build a more one-size-fits-all "base" image which you reference for all your services, or you have multiple base images with different more tailored sets of dependencies built up, or you have a very thin base image (e.g. not much more than ubuntu-latest) and just add what you need inside each individual Dockerfile. I ran into the same issue at first when laying out the Dockerfile hierarchy for all my services/roles, but I think in this case it actually falls under "less magic is better" / KISS. Dockerfiles are so easy to read and write once you get going, you won't mind the simplistic approach.
- ollysb 12y ago
  It sounds like you need something else to help you build the dockerfiles then i.e. I need a dockerfile with nginx, postgres and memcached so I use something like puppet/chef to put it together, is that possible?
  - nl 12y ago
    It's easy to build new images from Dockerfiles. And "cut-n-paste" based "inheritance" works well when combining two or more Dockerfiles into a single one that has everything you want. But you can't use composition to combine the actual downloaded images themselves. Using Chef within Docker to build up the services is documented on the Docker site[1], and of course puppet or whatever is essentially the same. [1] http://docs.docker.com/articles/chef/ http://docs.docker.com/articles/chef/

vidarh 12y ago

> For example, if I want a Ruby image that has the latest nginx and git installed, I can't just blend three images together. I have to pick a base image, and then manually add the rest myself via Dockerfile. My solution to this is to avoid combining stuff in the same image. E.g. my home server has an image for haproxy that splits incoming requests to my various test web apps by hostname. Each of those web apps run in their own docker container that bind-mount their appropriate directory from my home dir. Then the container(s) I do my development in likewise bind-mount the appropriate directory. For the most part I ssh in to a single screen session, but for anything that has particular dependencies, I have separate containers (e.g. I keep a Ruby 1.8 container available to test code that should retain compatibility). So far, all of my Docker containers are extremely small for this reason: I compose by orchestrating multiple Docker containers, not by stuffing multiple things into each container. This for the most part means that dependencies I need to manually add rarely is more than installing an extra package with apt, or installing another gem. I agree with you that it's hard to find decent images to rely on. But by keeping the containers small and single purpose I find that it's not that often it matters - the generic part of the config is often a matter of a couple of lines, while the Dockerfile and associated config files etc. tend to be dominated by my personal requirements.

peterwwillis 12y ago

> For example, if I want a Ruby image that has the latest nginx and git installed, I can't just blend three images together. I have to pick a base image, and then manually add the rest myself via Dockerfile. Wouldn't it be just crazy if someone maintained a list of software they built to be modularly installed along with other optional dependencies compiled on a common architecture and operating system for a common base image? We could make a tool to download them from a list of approved sources and verify that the files were cryptographically signed by a particular builder. We could also have the dependent images downloaded automatically. Then we could extend scripting support to run custom-built tools to tailor the install to what was on the system at the time of install or uninstall. Heck, considering all that they could make it cross-platform or even multi-platform. Maybe they could call it the Really Professional-docker-image Manager, or the Docker Professional Knowledge Gem, or the Yodeler's Ultimate Maintainer package. That would be crazy. Somebody should create a startup for this ASAP. [note to NSA crawlers: this has been an example of sarcasm]

asb 12y ago

It seems a shame that Ubuntu's Juju has been passed over in this space. Their charm store is designed to make it easy to mix and match recipes, have the community endorse and improve them. See the 'features' tab on https://jujucharms.com/sidebar/search/precise/postgresql-71/?text=postgresql#summary https://jujucharms.com/sidebar/search/precise/postgresql-71/... for a nice summary of what is offered by the most popular postgres charm.