11 ms·

Brief thoughts on Docker

92 points by craigkerstiens 13y ago

WestCoastJustin 13y ago
There are a bunch of cool and exciting things about Docker and containers in general. Docker just automates many manual tasks, essentially creating an abstraction layer above many manual and tricky steps. Similar to how Vagrant abstracted away many tasks with automating virtual machine deployment. There are a couple really cool things though, such as the Docker push/pull functionality, giving you access to Docker images, Vagrant does something similar to Vagrant boxes, but dockers goes a little further. But the killer idea, is that you do not need an OS in these Docker containers. You can basically have a statically linked binary, or a rails app, or sshd daemon, and that's it. You can then use cgroups to finely tune your resource levels (cpu, memory, bandwidth, etc) on this container. These concepts have been around since at least 2008 (LXC [1], was released into 2.6.24 [2]), Docker just streamlines the process greatly. As a side note, it looks like Google has been heavily using Linux containers for years! There is a Wired article [3], which talks about the orchestration engine they use to deploy containers across their cells. Google's John Wilkes even does a talk, entitle "Cluster management at Google" [4], where he discusses the next generation of the orchestration engine. p.s. It looks like there might even be "live container migration" in the pipeline [5]. This would allow you to move a container from machineA to machineB without any downtime, similar to Xen migrate or VMware live motion. [1] http://en.wikipedia.org/wiki/LXC http://en.wikipedia.org/wiki/LXC [2] http://kernelnewbies.org/Linux_2_6_24 http://kernelnewbies.org/Linux_2_6_24 [3] http://www.wired.com/wiredenterprise/2013/03/google-borg-twitter-mesos/all/ http://www.wired.com/wiredenterprise/2013/03/google-borg-twi... [4] http://www.youtube.com/watch?v=0ZFMlO98Jkc http://www.youtube.com/watch?v=0ZFMlO98Jkc [5] https://www.youtube.com/watch?feature=player_detailpage&v=LDhrDpz8JQw&t=3624 https://www.youtube.com/watch?feature=player_detailpage&v=LD...
- eikenberry 13y ago
  But the killer idea, is that you do not need an OS in these Docker containers. You can basically have a statically linked binary, or a rails app, or sshd daemon, and that's it. Is there documentation of this somewhere? I thought docker relied on having an operating system installed, even if just a minimal busybox based system.
  - WestCoastJustin 13y ago
    I'm not sure if there is documentation at this time, but there was a Google Hangout, entitled "Docker: the Linux container runtime" [1] (minute 24), where Solomon Hykes and Jérôme Petazzoni (from dotCloud, makers of Docker) talk about this very thing. [1] https://www.youtube.com/watch?feature=player_detailpage&v=LDhrDpz8JQw&t=1436 https://www.youtube.com/watch?feature=player_detailpage&v=LD...
  - andrewflnr 13y ago
    A docker container runs in an operating system, but doesn't contain one.
    - FooBarWidget 13y ago
      Well, that depends on your definition of "operating system". Is an OS just the kernel? Then yes, Docker doesn't contain an OS. Is the OS the userland? E.g. the init process + the shell + the C library + other runtimes? Then most Docker containers actually do contain an OS. Yes you can get rid of most of them by making static binaries, but why would you want to? Making static binaries with anything besides Go is a pain, and good luck trying to statically link MySQL or Redis into your binary.
      - icebraining 13y ago
        I don't think you need the init process even without a statically linked binary, and I don't see why would having a bunch of libraries suddenly make it an OS. The linking happens "outside" the container - in the shared kernel.
        justincormack 13y ago
        I think docker takes the place of init in the container, so no.
      - WestCoastJustin 13y ago
        You are totally correct. Lets take sshd for example, you would need to create a skeleton directory structure, /bin, /dev, /lib, etc in your rootfs container, have /bin/bash, a whack of /libs, some /dev devices, and then your sshd application. So an extremely minimalistic rootfs with only the sshd requirements. You could find these my running 'lsof -p `pidof sshd`' , maybe 'ldd /usr/sbin/sshd' (and all their dependencies) too, which quickly snowballs. Hopefully this explains where I was going with that. Also, this is not a simple task to try and strip off these services into self contained entities. There are lots of hidden issues, like how do we handle logging? Should the container have syslog too? So, there is work that needs to happen, I just like the idea of not running a full fledged OS in a container.
- sarnowski 13y ago
  FreeBSD jails already provided those containers for over 10 years. Even chroot is some kind of container. http://en.m.wikipedia.org/wiki/Operating_system-level_virtualization#Implementations http://en.m.wikipedia.org/wiki/Operating_system-level_virtua...
simlevesque 13y ago
Could someone explain to me the difference between Docker and Vagrant ?
- WestCoastJustin 13y ago
  Vagrant manages virtual machines. Docker uses linux containers, which is essentially "chroot on steroids" [1]. A linux container is just an isolated process (from which you run your code) on the host machine, like a FreeBSD jail. The host machine can peer into the container, but the container cannot see the host machine. Docker just helps to automate and manages the linux container life-cycle. p.s. I'm planning on doing a couple screencast episodes about linux containers and docker on my website, which should be online in the next week or so @ http://sysadmincasts.com/ http://sysadmincasts.com/ [1] http://lxc.sourceforge.net/ http://lxc.sourceforge.net/
  - patrickaljord 13y ago
    Please post your screencasts here.
    - arms 13y ago
      Seconded. I'm looking for as much learning material on Docker as possible.
- frio 13y ago
  They're actually quite similar. Docker is to lightweight containers as Vagrant is to virtual machines (more or less anyway; Vagrant lets you configure a lot more at this point). The nice thing is that the Dockerfile makes the whole thing a lot more transparent; VMs are sort of nebulous black boxes, whereas you can follow the construction of the container through the Dockerfile. I'll be interested in seeing how the ecosystem evolves when Vagrant becomes better able to manage containers.
  - aeon10 13y ago
    There is also a lot of talk about Docker and Vagrant working together. Like using Vagrant to manage Docker containers..
FooBarWidget 13y ago
Docker is great, but there is one security issue that it should solve. Right now, any user can manipulate any Docker container. If you create a container as root, then www-data can kill that container. The reason for this is because container orchestration is done through the Docker daemon, which listens on a TCP socket. Anybody on the local host can access that socket. They should, at the very least, implement password protection. Today I also found out that it's not possible to run Docker inside Docker. :( I'm working on a Docker-based continuous integration system similar to Travis, and it would be great if I can distribute the CI system as a Docker container.
- icebraining 13y ago
  You can actually configure Docker to use an Unix socket instead of TCP[1]. It seems it still opens it up to every user (it runs a chmod 777 on it), but you can always enforce stricter controls (e.g. with SELinux). [1] https://github.com/dotcloud/docker/pull/938 https://github.com/dotcloud/docker/pull/938
- jtgeibel 13y ago
  I came across this wiki page last night. It doesn't support this out of the box, but it looks like there is a way to run docker inside of a docker container. It seems to take a good bit of customization though. https://github.com/dotcloud/docker/wiki/Docker-in-Docker https://github.com/dotcloud/docker/wiki/Docker-in-Docker
- kstaken 13y ago
  Docker will be changing to just listen on a unix domain socket by default.
agumonkey 13y ago
I don't know for memory usage (although I'd bet containers uses less) but a VM take time to boot, with docker it's literally up in a second.
ghayes 13y ago
I've been reading about Docker, Flynn and CoreOS on HN for the last several days. Would anyone care to venture how these technologies might work together? That is, could Flynn load CoreOS VMs/AMIs on a cloud service which hosts Docker containers? Does CoreOS etcd allow these VMs/containers to self-configure?
- kstaken 13y ago
  CoreOS is an operating system designed to run docker containers. You would use it as lighter weight replacement for something like Ubuntu as the base os that Docker runs on. When you install CoreOS you basically just get a kernel, Docker and etcd plus a minimal number of other processes. Etcd is designed as a way to allow a distributed set of containers to self-configure. Flynn is an open source project to build a "platform as a service" platform on top of Docker. In theory Flynn could run on top of CoreOS. Docker is the core of everything.
deleted 13y ago
[deleted]
deleted 13y ago
[deleted]
relaxitup 13y ago
Are there any PaaS or VPS providers that are using Docker (other than DotCloud) ? I'm actually eager to give openstack-docker a testdrive.
- fmdud 13y ago
  If a PaaS/VPS provider is actually USING Docker to provide their service, it probably wouldn't be the best platform to actually TRY Docker in. It's easy enough to get it set up on DigitalOcean; I created a tutorial at https://www.digitalocean.com/community/articles/how-to-install-docker-on-ubuntu-13-04-x64-vps https://www.digitalocean.com/community/articles/how-to-insta...
  - relaxitup 13y ago
    Actually my second sentence was not connected to the first. I was curious if there are any PaaS providers using Docker as the underlying infrastructure. And I also would like to setup openstack-docker and play around with it (not inside a PaaS or VPS of course). I did see your tutorials though, definitely another great way to test out Docker quickly.
- julien421 13y ago
  Hello Relaxitup, There are several projects to build a PaaS around Docker: - Deis: https://pypi.python.org/pypi/deis/0.0.4 https://pypi.python.org/pypi/deis/0.0.4 - Dokku: https://github.com/progrium/dokku https://github.com/progrium/dokku - Flynn: https://flynn.io/ https://flynn.io/ Also, have a look at this blog post for playing with openstack + docker: http://blog.docker.io/2013/06/openstack-docker-manage-linux-containers-with-nova/ http://blog.docker.io/2013/06/openstack-docker-manage-linux-...
  - relaxitup 13y ago
    Thanks julien421 will check these out.
boothead 13y ago
Anyone looked at combining something like ansible and docker for building images? It's pretty straightforward to use ansible to build the host. This is one step forward but I wonder if anyone's used it to create images and if so how? Would you build on top a base that includes the ansible libraries, or somehow run the ansible commands against a docker instance?

There are a bunch of cool and exciting things about Docker and containers in general. Docker just automates many manual tasks, essentially creating an abstraction layer above many manual and tricky steps. Similar to how Vagrant abstracted away many tasks with automating virtual machine deployment. There are a couple really cool things though, such as the Docker push/pull functionality, giving you access to Docker images, Vagrant does something similar to Vagrant boxes, but dockers goes a little further. But the killer idea, is that you do not need an OS in these Docker containers. You can basically have a statically linked binary, or a rails app, or sshd daemon, and that's it. You can then use cgroups to finely tune your resource levels (cpu, memory, bandwidth, etc) on this container. These concepts have been around since at least 2008 (LXC [1], was released into 2.6.24 [2]), Docker just streamlines the process greatly. As a side note, it looks like Google has been heavily using Linux containers for years! There is a Wired article [3], which talks about the orchestration engine they use to deploy containers across their cells. Google's John Wilkes even does a talk, entitle "Cluster management at Google" [4], where he discusses the next generation of the orchestration engine. p.s. It looks like there might even be "live container migration" in the pipeline [5]. This would allow you to move a container from machineA to machineB without any downtime, similar to Xen migrate or VMware live motion. [1] http://en.wikipedia.org/wiki/LXC http://en.wikipedia.org/wiki/LXC [2] http://kernelnewbies.org/Linux_2_6_24 http://kernelnewbies.org/Linux_2_6_24 [3] http://www.wired.com/wiredenterprise/2013/03/google-borg-twitter-mesos/all/ http://www.wired.com/wiredenterprise/2013/03/google-borg-twi... [4] http://www.youtube.com/watch?v=0ZFMlO98Jkc http://www.youtube.com/watch?v=0ZFMlO98Jkc [5] https://www.youtube.com/watch?feature=player_detailpage&v=LDhrDpz8JQw&t=3624 https://www.youtube.com/watch?feature=player_detailpage&v=LD...

eikenberry 13y ago
But the killer idea, is that you do not need an OS in these Docker containers. You can basically have a statically linked binary, or a rails app, or sshd daemon, and that's it. Is there documentation of this somewhere? I thought docker relied on having an operating system installed, even if just a minimal busybox based system.
- WestCoastJustin 13y ago
  I'm not sure if there is documentation at this time, but there was a Google Hangout, entitled "Docker: the Linux container runtime" [1] (minute 24), where Solomon Hykes and Jérôme Petazzoni (from dotCloud, makers of Docker) talk about this very thing. [1] https://www.youtube.com/watch?feature=player_detailpage&v=LDhrDpz8JQw&t=1436 https://www.youtube.com/watch?feature=player_detailpage&v=LD...
- andrewflnr 13y ago
  A docker container runs in an operating system, but doesn't contain one.
  - FooBarWidget 13y ago
    Well, that depends on your definition of "operating system". Is an OS just the kernel? Then yes, Docker doesn't contain an OS. Is the OS the userland? E.g. the init process + the shell + the C library + other runtimes? Then most Docker containers actually do contain an OS. Yes you can get rid of most of them by making static binaries, but why would you want to? Making static binaries with anything besides Go is a pain, and good luck trying to statically link MySQL or Redis into your binary.
    - icebraining 13y ago
      I don't think you need the init process even without a statically linked binary, and I don't see why would having a bunch of libraries suddenly make it an OS. The linking happens "outside" the container - in the shared kernel.
      - justincormack 13y ago
        I think docker takes the place of init in the container, so no.
    - WestCoastJustin 13y ago
      You are totally correct. Lets take sshd for example, you would need to create a skeleton directory structure, /bin, /dev, /lib, etc in your rootfs container, have /bin/bash, a whack of /libs, some /dev devices, and then your sshd application. So an extremely minimalistic rootfs with only the sshd requirements. You could find these my running 'lsof -p `pidof sshd`' , maybe 'ldd /usr/sbin/sshd' (and all their dependencies) too, which quickly snowballs. Hopefully this explains where I was going with that. Also, this is not a simple task to try and strip off these services into self contained entities. There are lots of hidden issues, like how do we handle logging? Should the container have syslog too? So, there is work that needs to happen, I just like the idea of not running a full fledged OS in a container.
sarnowski 13y ago
FreeBSD jails already provided those containers for over 10 years. Even chroot is some kind of container. http://en.m.wikipedia.org/wiki/Operating_system-level_virtualization#Implementations http://en.m.wikipedia.org/wiki/Operating_system-level_virtua...

simlevesque 13y ago

Could someone explain to me the difference between Docker and Vagrant ?

WestCoastJustin 13y ago
Vagrant manages virtual machines. Docker uses linux containers, which is essentially "chroot on steroids" [1]. A linux container is just an isolated process (from which you run your code) on the host machine, like a FreeBSD jail. The host machine can peer into the container, but the container cannot see the host machine. Docker just helps to automate and manages the linux container life-cycle. p.s. I'm planning on doing a couple screencast episodes about linux containers and docker on my website, which should be online in the next week or so @ http://sysadmincasts.com/ http://sysadmincasts.com/ [1] http://lxc.sourceforge.net/ http://lxc.sourceforge.net/
- patrickaljord 13y ago
  Please post your screencasts here.
  - arms 13y ago
    Seconded. I'm looking for as much learning material on Docker as possible.
frio 13y ago
They're actually quite similar. Docker is to lightweight containers as Vagrant is to virtual machines (more or less anyway; Vagrant lets you configure a lot more at this point). The nice thing is that the Dockerfile makes the whole thing a lot more transparent; VMs are sort of nebulous black boxes, whereas you can follow the construction of the container through the Dockerfile. I'll be interested in seeing how the ecosystem evolves when Vagrant becomes better able to manage containers.
- aeon10 13y ago
  There is also a lot of talk about Docker and Vagrant working together. Like using Vagrant to manage Docker containers..

FooBarWidget 13y ago

Docker is great, but there is one security issue that it should solve. Right now, any user can manipulate any Docker container. If you create a container as root, then www-data can kill that container. The reason for this is because container orchestration is done through the Docker daemon, which listens on a TCP socket. Anybody on the local host can access that socket. They should, at the very least, implement password protection. Today I also found out that it's not possible to run Docker inside Docker. :( I'm working on a Docker-based continuous integration system similar to Travis, and it would be great if I can distribute the CI system as a Docker container.

icebraining 13y ago
You can actually configure Docker to use an Unix socket instead of TCP[1]. It seems it still opens it up to every user (it runs a chmod 777 on it), but you can always enforce stricter controls (e.g. with SELinux). [1] https://github.com/dotcloud/docker/pull/938 https://github.com/dotcloud/docker/pull/938
jtgeibel 13y ago
I came across this wiki page last night. It doesn't support this out of the box, but it looks like there is a way to run docker inside of a docker container. It seems to take a good bit of customization though. https://github.com/dotcloud/docker/wiki/Docker-in-Docker https://github.com/dotcloud/docker/wiki/Docker-in-Docker
kstaken 13y ago
Docker will be changing to just listen on a unix domain socket by default.

agumonkey 13y ago

I don't know for memory usage (although I'd bet containers uses less) but a VM take time to boot, with docker it's literally up in a second.

ghayes 13y ago

I've been reading about Docker, Flynn and CoreOS on HN for the last several days. Would anyone care to venture how these technologies might work together? That is, could Flynn load CoreOS VMs/AMIs on a cloud service which hosts Docker containers? Does CoreOS etcd allow these VMs/containers to self-configure?

kstaken 13y ago
CoreOS is an operating system designed to run docker containers. You would use it as lighter weight replacement for something like Ubuntu as the base os that Docker runs on. When you install CoreOS you basically just get a kernel, Docker and etcd plus a minimal number of other processes. Etcd is designed as a way to allow a distributed set of containers to self-configure. Flynn is an open source project to build a "platform as a service" platform on top of Docker. In theory Flynn could run on top of CoreOS. Docker is the core of everything.

deleted 13y ago

[deleted]

deleted 13y ago

[deleted]

relaxitup 13y ago

Are there any PaaS or VPS providers that are using Docker (other than DotCloud) ? I'm actually eager to give openstack-docker a testdrive.

fmdud 13y ago
If a PaaS/VPS provider is actually USING Docker to provide their service, it probably wouldn't be the best platform to actually TRY Docker in. It's easy enough to get it set up on DigitalOcean; I created a tutorial at https://www.digitalocean.com/community/articles/how-to-install-docker-on-ubuntu-13-04-x64-vps https://www.digitalocean.com/community/articles/how-to-insta...
- relaxitup 13y ago
  Actually my second sentence was not connected to the first. I was curious if there are any PaaS providers using Docker as the underlying infrastructure. And I also would like to setup openstack-docker and play around with it (not inside a PaaS or VPS of course). I did see your tutorials though, definitely another great way to test out Docker quickly.
julien421 13y ago
Hello Relaxitup, There are several projects to build a PaaS around Docker: - Deis: https://pypi.python.org/pypi/deis/0.0.4 https://pypi.python.org/pypi/deis/0.0.4 - Dokku: https://github.com/progrium/dokku https://github.com/progrium/dokku - Flynn: https://flynn.io/ https://flynn.io/ Also, have a look at this blog post for playing with openstack + docker: http://blog.docker.io/2013/06/openstack-docker-manage-linux-containers-with-nova/ http://blog.docker.io/2013/06/openstack-docker-manage-linux-...
- relaxitup 13y ago
  Thanks julien421 will check these out.

boothead 13y ago

Anyone looked at combining something like ansible and docker for building images? It's pretty straightforward to use ansible to build the host. This is one step forward but I wonder if anyone's used it to create images and if so how? Would you build on top a base that includes the ansible libraries, or somehow run the ansible commands against a docker instance?