12 ms·

Docker 0.8: Quality, new builder features, btrfs, OSX support

291 points by asb 13y ago

arc_of_descent 13y ago
I just tried the docker interactive tutorial. It was fun, but I still don't get the point of using docker. Just been hearing a lot about it, and its getting too much buzz.
mattbessey 13y ago
Glad to hear OS X has official support. I jumped into Docker for the first time last week and have a burning unresolved question for those using boot2docker. What is your development workflow? I am working on a Rails app, so my instinct is to have a shared folder between OS X and boot2docker, but afaik this is not supported as boot2docker doesn't support VirtualBox guest extensions.
- shykes 13y ago
  Hi Matt, you are not alone :) It turns out that shared folders are not a sustainable solution (independently of whether boot2docker supports them), so the best practices are converging towards this: 1) While developing, your dev environment (including the source code and method for fetching it) should live in a container. This container could be as simple as a shell box with git and ssh installed, where you keep a terminal open and run your unit tests etc. 2) To access your source code on your host machine (eg. for editing on your mac), export it from your container over a network filesystem: samba, nfs or 9p are popular examples. Then mount that from your mac. Samba can be natively mounted with "command-K". NFS and 9p require macfuse. 3) When building the final container for integration tests, staging and production, go through the full Dockerfile + 'docker build' process. 'docker build' on your mac will transparently upload the source over the docker remote API as needed. There are several advantages to exporting the source from the container to the host, instead of the other way around: - It's less infrastructure-specific. If you move from virtualbox to vmware, or get a Linux laptop and run docker straight on the metal, your storage/shared folders configuration doesn't change: all you need is a network connection to the container. - Network filesystems are more reliable than shared folders + bind-mount. For example they can handle different permissions and ownership on both ends - a very common problem with shared folders is "oops the container creates files as root but I don't have root on my mac", or "apache complains that the permissions are all wrong because virtualbox shared folders threw up on me". That said, we need to take that design insight and turn it into a polished user experience - hopefully in Docker 0.9 this will all be much more seamless!
  - bstar77 13y ago
    Thanks for taking the time to write this. I've hit a major wall in figuring out the best workflow for this exact scenario. Good to finally hear an official suggestion on the matter. I've been depending on shared directories, so I'll definitely be experimenting with network filesystems. As Docker evolves, it would be great to have some kind of official resource to get suggestions for optimal workflows as new features become available (the weekly docker email is my best resource right now). Searching the internet for info has been a huge chore as most of the resources (including the ones hosted by docker.io) are woefully out of date.
    - shykes 13y ago
      > As Docker evolves, it would be great to have some kind of official resource to get suggestions for optimal workflows as new features become available Yes! We are trying to figure this out. Our current avenue for this is to dedicate a new section in the docs to use cases and best practices. As you point out, our docs (and written content in general) are often inaccurate. We need to fix this. Hopefully in the coming weeks you will start seeing notable improvements in these areas. Thanks for bearing with us!
  - tksb 13y ago
    Thank you for this. I found it extremely helpful.
  - ecnahc515 13y ago
    This should be a blog post or tutorial or guide on the documentation page. It would be really helpful to have guidance on how to do development work flow using docker while avoiding bad practices like shared volumes.
    - FireBeyond 13y ago
      Seconded. My biggest stumbling block with Docker at the moment is “best practices”, as I work on coming up with a Docker Dev Ecosystem (for myself and for a team).
- odonnellryan 13y ago
  You can use Docker + Makefiles (test, binary, etc) to really help you with this.
shykes 13y ago
Just to clarify on the OSX support: obviously we did not magically get Darwin to support linux containers. But we put together the easiest possible way to run Linux containers on a Mac without depending on another machine. We do this by combining (1) docker in "client mode", which connects to (2) a super-lightweight linux VM using boot2docker. The details are on http://docs.docker.io/en/latest/installation/mac/ http://docs.docker.io/en/latest/installation/mac/
- steeve 13y ago
  Note that you need to be running boot2docker 0.5.2 [1] (with docker 0.8) for docker to work properly on OS X. [1] https://github.com/steeve/boot2docker/releases/tag/v0.5.2 https://github.com/steeve/boot2docker/releases/tag/v0.5.2
- oellegaard 13y ago
  Too bad it depends on VirtualBox - had a lot of kernel panics when using it, so I decided to stick with VMware fusion.
  - ARothfusz 13y ago
    You should be able to install the same iso into a VMware image. You'll need to expose the networking ports, but the information should be in the boot2docker script.
    - tianon 13y ago
      Or even bare metal. Personally, I extract the kernel and initrd and use it directly via PXE.
      - lengads 13y ago
        Is that to say that you.. wait, what are you doing?
        13y ago
davidcelis 13y ago
Awesome that OSX support is now official, but is there any benefit to using this process as opposed to using docker-osx? https://github.com/noplay/docker-osx https://github.com/noplay/docker-osx The official installation process seems more complicated, and I don't really see an advantage.
1qaz2wsx3edc 13y ago
It's not explicitly said, but are they following Semantic Versioning? http://semver.org/ http://semver.org/
- skywhopper 13y ago
  Not exactly. The article says the first number is for major lifecycle events, ie 1.0 means "production ready". They'll be releasing monthly and the second number will be the release increment. The third number will be for patches and fixes. So to me that doesn't fit in with the Semantic Versioning contract. I think the product is too young yet to use a version scheme that assumes relative API stability.
  - stdbrouw 13y ago
    Well, to be fair, this is why even with semver 0.x means "anything goes". It's only from 1.x onwards that major version increments should be used for backwards-incompatible changes.
  - 1qaz2wsx3edc 13y ago
    Well, I would assume 0.x.x is by nature unstable, so I'm not sure how it doesn't fit. I read the article, but I'm still left wondering for clarification.
steeve 13y ago
Note that you need to be running boot2docker 0.5.2 [1] (with docker 0.8) for docker to work properly on OS X. [1] https://github.com/steeve/boot2docker/releases/tag/v0.5.2 https://github.com/steeve/boot2docker/releases/tag/v0.5.2
ak217 13y ago
I'm curious about how the focus on multiple, ABI-incompatible platforms will affect the pace and momentum of Docker development. So far, Docker has benefitted a lot from the focus on amd64 userland on Linux.
- tadfisher 13y ago
  Docker on OSX is actually running on a lightweight Linux VM, so it's not really multi-platform support they've added.
- derefr 13y ago
  Personally, when I read "OSX support", I thought that meant that there would now be containers with Darwin-ABI binaries inside them. So on Linux, you'd use cgroups for Linux-ABI binaries and a VM for Darwin-ABI, just as on OSX you use a VM for Linux-ABI (and presumably would use the OSX sandbox API for Darwin-ABI containers.) This "native sandboxing for own-ABI if available, VM if not, and VM for everything else" approach would extend to any other platform as well, I'd think (Windows, for example.) I'm surprised that this isn't where Docker is going, at least for development and testing of containers. (Though another alternative, probably more performant for production, would be something like having versions of CoreOS for each platform--CoreOS/Linux, CoreOS/Darwin, CoreOS/NT, and so on--so you'd have a cluster of machines with various ABIs, where any container you want to run gets shipped off to a machine in the cluster with the right ABI for it.)
  - FooBarWidget 13y ago
    Going that way would dilute Docker's value. Docker's promise is that you can build a container and it will always work; it won't mysteriously break in production or giving you installation headaches. To do that, your development environment has to be as close to the production environment as possible. Having a totally different ABI doesn't help with that goal.
    - shykes 13y ago
      Our priority in the short term is definitely to focus on the Linux ABI and making it available on as many physical machines as possible. This is the reasoning behind our current OSX support, and support for more platforms coming soon. Longer term we do need to support multiple ABIs, if only because a lot of people want to use Docker on x86-32 and ARM. Having ELF binaries built on Linux isn't of much help if they're built for another arch :) So at the very least we will need to support 3 ABIs in the near future. The good news is that it can be done in a way which doesn't hurt the repeatability of Docker's execution environment. Think of it this way: every container has certain requirements to run. At the very least it needs a certain range of kernels and archs (and yes it's possible, although uncommon for a binary to support multiple archs). It may also require a network interface to bind a socket on a certain TCP port. It may require certain syscalls to be authorized. It may require the ability to create a tun/tap device. And so on. Docker's job is to offer a portable abstraction for these requirements, so that the container can list what it needs on the one hand, the host can list what it offers on the other, and docker can match them in the middle. If the requirements listed by a given container aren't met ("I need CAP_SYSADMIN on a 3.8 linux kernel and an ARM processor!") then docker returns an error and a clear error message. If they are met, the container is executed and must always be repeatable. TLDR: ABI requirements are just one kind of requirements. Docker can handle multiple requirements without breaking the repeatability of its execution environment.
deleted 13y ago
[deleted]
joeshaw 13y ago
Was the OS X binary built without cgo? I can't seem to access containers in private https registries: $ docker login https://registry.example.com 2014/02/05 14:36:20 Invalid Registry endpoint: Get https://registry.example.com/v1/_ping: x509: failed to load system roots and no roots provided The hostname in question has a valid SSL certificate. I encountered a similar problem in the past with Go built from homebrew[1][2]. Has anyone else seen this? [1] https://github.com/Homebrew/homebrew/pull/17758 https://github.com/Homebrew/homebrew/pull/17758 [2] https://code.google.com/p/go/issues/detail?id=4791 https://code.google.com/p/go/issues/detail?id=4791 Update: Filed a bug against docker, others are having the same issue. https://github.com/dotcloud/docker/issues/3946 https://github.com/dotcloud/docker/issues/3946
- crosbymichael 13y ago
  Thanks for the report. We are working on a fix ASAP and will post updates on https://github.com/dotcloud/docker/issues/3683 https://github.com/dotcloud/docker/issues/3683.
  - jschorr 13y ago
    In the meantime, if you need to get it working right now, you can build your own binary as outlined here: http://blog.devtable.com/2014/01/using-docker-on-osx-with-private.html http://blog.devtable.com/2014/01/using-docker-on-osx-with-pr... We've confirmed the instructions still work with Docker 0.8 (make sure to change the checkout branch though :))
sbt 13y ago
I am interested in the BTRFS support in particular, it is clear that performance in FS is key. However, what I like the most about Docker is the ability to use layers and diff them. In effect, I want version control for images, because it allows me to not run extra provisioning tools for the images (just rely on simply Zookeeper stuff for app config). Whatever gives me 'vcs' for images in the most performant way, wins in my book.
jokoon 13y ago
can someone explain what this is ? and what's the purpose ?
- jamtur01 13y ago
  Try here: http://www.docker.io/learn_more/ http://www.docker.io/learn_more/
  - jokoon 13y ago
    nope, I still don't get it. > Docker is an open-source engine that automates the deployment of any application as a lightweight, portable, self-sufficient container that will run virtually anywhere. Like an internet browser for executables ? I don't understand how can this be useful...
    - McGlockenshire 13y ago
      Containers are like virtual machines, only without the hardware emulation. See http://en.wikipedia.org/wiki/LXC http://en.wikipedia.org/wiki/LXC See http://docs.docker.io/en/latest/faq/#how-do-containers-compare-to-virtual-machines http://docs.docker.io/en/latest/faq/#how-do-containers-compa... Docker is a mechanism to bundle an application together inside a container (think VM instance) in a way that makes it easier to distribute.
    - ecnahc515 13y ago
      Think of it like this: Say you have a python/rails/nodejs/c++/whatever app. Sometimes getting all the dependencies on the system is cumbersome and hard to manage. This is true for both developers, and the people deploying these apps. I can't count how many times I've had a python app fail to build on a new box because of some C extension and forgetting to install a package on (centOS, ubuntu, debian, ect). Docker lets me do all of this once for my app, with a Docker image, and now when I want to deploy it, all the system needs is Docker installed. This means on my laptop, on our staging server, on our production server, all they need is Docker, and I will have the exact same environment in each place, and to deploy the app, its exactly the same. There's a ton of other bonuses like each container's dependencies and processes are isolated from each other. I also get a ton of the Docker features that allow containers to communicate with each other and setup service discovery between them (ex: your database container can now expose information to the app container using environmental variables). Tons of other good reasons too though, you should check it out.
stesch 13y ago
Is Docker a good way to bring more security on a server with a few different websites? Separating the sites from each other and run nginx as a proxy in front of them? What's the overhead?
- zobzu 13y ago
  its not as good as using separate vms, but has arguably lower overhead also, its not really docker doing that, its LXC. Docker is an API around it.
  - nickstinemates 13y ago
    Docker is not an API around LXC. Any more than any application using a framework is a wrapper around said framework. However, the analogy I just made is only at a point in time. LXC is not the only Linux provider to manage cgroups/namespaces, it was just the most convenient at the time for the target audience. That is a fleeting position soon remedied. If you'd like to know more, I'd encourage you to get involved with Docker development.
    - zobzu 13y ago
      yeah, docket is an api around namespaces.. and thus lxc, right now. thanks for nothing really.
- FireBeyond 13y ago
  Yes, it is. It’s atop LXC (Linux Containers), which - oversimplified - is process namespacing, jails, etc. There’s near zero overhead, because there’s no virtualization.
- midas007 13y ago
  It depends. If the goal is to consolidate several boxes to a single VM, it does this. Be sure host-based (on the linux box) firewall rules are set and documented. If possible, set network-based firewall rules also (AWS security groups).
- _wmd 13y ago
midas007 13y ago
It's confusing why btrfs support was prioritized ahead of zfs considering zfs' superior architecture and ops capabilities. Is docker (formerly dotcloud) going to start withholding capabilities as licensed features? Edit: prelim zfs driver work is here https://github.com/gurjeet/docker/tree/zfs_driver https://github.com/gurjeet/docker/tree/zfs_driver
- tacticus 13y ago
  What superior architecture does zfs have?
  - rch 13y ago
    My understanding is that btrfs is still in catchup mode for the foreseeable future, but might eventually cover the distance. Has btrfs jumped ahead of zfs in ways I haven't heard about? Edit - this is my first search result: http://rudd-o.com/linux-and-free-software/ways-in-which-zfs-is-better-than-btrfs http://rudd-o.com/linux-and-free-software/ways-in-which-zfs-...
    - midas007 13y ago
      I'd be biased to agree since that's Manuel's blog, someone I used to work with. I've supported 24x7 and 9x5 ops where downtime was unacceptable. zfs makes it a whole lot easier to perform upgrades, know data and metadata are solid and send snapshots around.
      - rch 13y ago
        > ZFS uses atomic writes and barriers This about settles the question for me. Assuming that the implication that btrfs performs otherwise holds true.
        midas007 13y ago
        Yeah, it depends on the use case. For home directories and large risk items like financial stuff, testing that barrier writes are happening is a good thing. You don't want a storm to knock out a DC to learn that the hw/sw fs stack was lying to you at some level.
brunoqc 13y ago
Anyone using Docker on 32-bits?
- jamtur01 13y ago
  Docker does not (yet) support 32-bit architectures.
  - brunoqc 13y ago
    I think I read that you just have to disable a condition somewhere and build your own base image or something like that.
newman314 13y ago
IMO, it would be fantastic if there was something like Docker for Windows. Imagine being able to bundle up games in individual containers and easily being able to move them from machine to machine as you upgrade. Same thing applies for other Windows apps.
- smtddr 13y ago
  The developers of DRM for Windows games would probably make sure that doesn't work.
  - jamtur01 13y ago
    Not if it's an option for installation. Or if they start shipping them like that. Steam springs to mind. :)
- wslh 13y ago
  Docker is to Unix almost as Microsoft App-V/VMware ThinApp/Symantec Workspace Virtualization products are to Windows. Not exactly the same but closer.
  - newman314 13y ago
    Precisely, it's not the same. There still a whole bunch of fiddling going on. I'll stick to my statement. I want something Docker-like for Windows so that I can easily move things from one machine to another.
    - wslh 13y ago
      It seems like they are working on it: "Microsoft Corporation : Patent Issued for Extensible Application Virtualization Subsystems" http://www.4-traders.com/MICROSOFT-CORPORATION-4835/news/Microsoft-Corporation--Patent-Issued-for-Extensible-Application-Virtualization-Subsystems-17940954/ http://www.4-traders.com/MICROSOFT-CORPORATION-4835/news/Mic...
optymizer 13y ago
So what's the solution for 'root inside a docker container is root on the host'? We'd like to ship a set of utilities as a docker container, but unless the sysadmin gives everyone 'sudo' privileges on the server (unlikely and insecure), they can't run the container and its utilities. Any advice?
- shykes 13y ago
  Look for an update on this in 0.9 :) Future versions of the Docker API will natively support scoping. This means that each API client will see a different subset of the Docker engine depending on the credentials and origin of the connection. This will be implemented in combination with introspection, which allows any container to open a connection to the Docker engine which started it. When you combine scoping and introspection, you get really cool scenarios. For example, let's say your utility is called "dockermeister". Each individual user could deploy his own copy of dockermeister, in a separate container. Each dockermeister container would in turn connect to Docker (via introspection), destroy all existing containers, and create 10 fresh redis containers (for reasons unknown). Because each dockermeister container is scoped, it can only remove containers that are its children (ie that were created from the same container at an earlier time). So they cannot affect each other. Likewise, the 10 new redis containers will only be visible to that particular user, and not pollute the namespace of the other users. Of course scoping works at arbitrary depth levels... so you could have containers starting containers starting containers. Containers all the way down :)
- kapilvt 13y ago
  user namespaces will solve this. lxc in ubuntu 14.04 already supports docker in user lxc, ie. no root needed. add silly disclaimer.. yes docker has some notion of portable containers plugins, but it uses lxc atm, and the feature is in lxc upstream.
andrewcooke 13y ago
why is osx support necessary on the path to 1.0? i'd rather have a simple, small, 1.0 release i can trust than all these "bells and whistles".
- shykes 13y ago
  It's not necessary and we didn't go out of our way to get it. We just happened "for free" as a result of writing portable code, a clean client-server architecture and the appearance of the boot2docker project in the community.
morgante 13y ago
Awesome release, especially since I can now use Docker on OS X without having to boot up a full Ubuntu VM through Vagrant. If anyone else is using Boxen, I packaged up a quick Puppet module to get up and running with Docker on OS X: https://github.com/morgante/puppet-docker https://github.com/morgante/puppet-docker

291 points by asb 13y ago

I just tried the docker interactive tutorial. It was fun, but I still don't get the point of using docker. Just been hearing a lot about it, and its getting too much buzz.

mattbessey 13y ago

Glad to hear OS X has official support. I jumped into Docker for the first time last week and have a burning unresolved question for those using boot2docker. What is your development workflow? I am working on a Rails app, so my instinct is to have a shared folder between OS X and boot2docker, but afaik this is not supported as boot2docker doesn't support VirtualBox guest extensions.

shykes 13y ago
Hi Matt, you are not alone :) It turns out that shared folders are not a sustainable solution (independently of whether boot2docker supports them), so the best practices are converging towards this: 1) While developing, your dev environment (including the source code and method for fetching it) should live in a container. This container could be as simple as a shell box with git and ssh installed, where you keep a terminal open and run your unit tests etc. 2) To access your source code on your host machine (eg. for editing on your mac), export it from your container over a network filesystem: samba, nfs or 9p are popular examples. Then mount that from your mac. Samba can be natively mounted with "command-K". NFS and 9p require macfuse. 3) When building the final container for integration tests, staging and production, go through the full Dockerfile + 'docker build' process. 'docker build' on your mac will transparently upload the source over the docker remote API as needed. There are several advantages to exporting the source from the container to the host, instead of the other way around: - It's less infrastructure-specific. If you move from virtualbox to vmware, or get a Linux laptop and run docker straight on the metal, your storage/shared folders configuration doesn't change: all you need is a network connection to the container. - Network filesystems are more reliable than shared folders + bind-mount. For example they can handle different permissions and ownership on both ends - a very common problem with shared folders is "oops the container creates files as root but I don't have root on my mac", or "apache complains that the permissions are all wrong because virtualbox shared folders threw up on me". That said, we need to take that design insight and turn it into a polished user experience - hopefully in Docker 0.9 this will all be much more seamless!
- bstar77 13y ago
  Thanks for taking the time to write this. I've hit a major wall in figuring out the best workflow for this exact scenario. Good to finally hear an official suggestion on the matter. I've been depending on shared directories, so I'll definitely be experimenting with network filesystems. As Docker evolves, it would be great to have some kind of official resource to get suggestions for optimal workflows as new features become available (the weekly docker email is my best resource right now). Searching the internet for info has been a huge chore as most of the resources (including the ones hosted by docker.io) are woefully out of date.
  - shykes 13y ago
    > As Docker evolves, it would be great to have some kind of official resource to get suggestions for optimal workflows as new features become available Yes! We are trying to figure this out. Our current avenue for this is to dedicate a new section in the docs to use cases and best practices. As you point out, our docs (and written content in general) are often inaccurate. We need to fix this. Hopefully in the coming weeks you will start seeing notable improvements in these areas. Thanks for bearing with us!
- tksb 13y ago
  Thank you for this. I found it extremely helpful.
- ecnahc515 13y ago
  This should be a blog post or tutorial or guide on the documentation page. It would be really helpful to have guidance on how to do development work flow using docker while avoiding bad practices like shared volumes.
  - FireBeyond 13y ago
    Seconded. My biggest stumbling block with Docker at the moment is “best practices”, as I work on coming up with a Docker Dev Ecosystem (for myself and for a team).
odonnellryan 13y ago
You can use Docker + Makefiles (test, binary, etc) to really help you with this.

shykes 13y ago

Just to clarify on the OSX support: obviously we did not magically get Darwin to support linux containers. But we put together the easiest possible way to run Linux containers on a Mac without depending on another machine. We do this by combining (1) docker in "client mode", which connects to (2) a super-lightweight linux VM using boot2docker. The details are on http://docs.docker.io/en/latest/installation/mac/ http://docs.docker.io/en/latest/installation/mac/

steeve 13y ago
Note that you need to be running boot2docker 0.5.2 [1] (with docker 0.8) for docker to work properly on OS X. [1] https://github.com/steeve/boot2docker/releases/tag/v0.5.2 https://github.com/steeve/boot2docker/releases/tag/v0.5.2
oellegaard 13y ago
Too bad it depends on VirtualBox - had a lot of kernel panics when using it, so I decided to stick with VMware fusion.
- ARothfusz 13y ago
  You should be able to install the same iso into a VMware image. You'll need to expose the networking ports, but the information should be in the boot2docker script.
  - tianon 13y ago
    Or even bare metal. Personally, I extract the kernel and initrd and use it directly via PXE.
    - lengads 13y ago
      Is that to say that you.. wait, what are you doing?
      - 13y ago

davidcelis 13y ago

Awesome that OSX support is now official, but is there any benefit to using this process as opposed to using docker-osx? https://github.com/noplay/docker-osx https://github.com/noplay/docker-osx The official installation process seems more complicated, and I don't really see an advantage.

1qaz2wsx3edc 13y ago

It's not explicitly said, but are they following Semantic Versioning? http://semver.org/ http://semver.org/

skywhopper 13y ago
Not exactly. The article says the first number is for major lifecycle events, ie 1.0 means "production ready". They'll be releasing monthly and the second number will be the release increment. The third number will be for patches and fixes. So to me that doesn't fit in with the Semantic Versioning contract. I think the product is too young yet to use a version scheme that assumes relative API stability.
- stdbrouw 13y ago
  Well, to be fair, this is why even with semver 0.x means "anything goes". It's only from 1.x onwards that major version increments should be used for backwards-incompatible changes.
- 1qaz2wsx3edc 13y ago
  Well, I would assume 0.x.x is by nature unstable, so I'm not sure how it doesn't fit. I read the article, but I'm still left wondering for clarification.

steeve 13y ago

Note that you need to be running boot2docker 0.5.2 [1] (with docker 0.8) for docker to work properly on OS X. [1] https://github.com/steeve/boot2docker/releases/tag/v0.5.2 https://github.com/steeve/boot2docker/releases/tag/v0.5.2

ak217 13y ago

I'm curious about how the focus on multiple, ABI-incompatible platforms will affect the pace and momentum of Docker development. So far, Docker has benefitted a lot from the focus on amd64 userland on Linux.

tadfisher 13y ago
Docker on OSX is actually running on a lightweight Linux VM, so it's not really multi-platform support they've added.
derefr 13y ago
Personally, when I read "OSX support", I thought that meant that there would now be containers with Darwin-ABI binaries inside them. So on Linux, you'd use cgroups for Linux-ABI binaries and a VM for Darwin-ABI, just as on OSX you use a VM for Linux-ABI (and presumably would use the OSX sandbox API for Darwin-ABI containers.) This "native sandboxing for own-ABI if available, VM if not, and VM for everything else" approach would extend to any other platform as well, I'd think (Windows, for example.) I'm surprised that this isn't where Docker is going, at least for development and testing of containers. (Though another alternative, probably more performant for production, would be something like having versions of CoreOS for each platform--CoreOS/Linux, CoreOS/Darwin, CoreOS/NT, and so on--so you'd have a cluster of machines with various ABIs, where any container you want to run gets shipped off to a machine in the cluster with the right ABI for it.)
- FooBarWidget 13y ago
  Going that way would dilute Docker's value. Docker's promise is that you can build a container and it will always work; it won't mysteriously break in production or giving you installation headaches. To do that, your development environment has to be as close to the production environment as possible. Having a totally different ABI doesn't help with that goal.
  - shykes 13y ago
    Our priority in the short term is definitely to focus on the Linux ABI and making it available on as many physical machines as possible. This is the reasoning behind our current OSX support, and support for more platforms coming soon. Longer term we do need to support multiple ABIs, if only because a lot of people want to use Docker on x86-32 and ARM. Having ELF binaries built on Linux isn't of much help if they're built for another arch :) So at the very least we will need to support 3 ABIs in the near future. The good news is that it can be done in a way which doesn't hurt the repeatability of Docker's execution environment. Think of it this way: every container has certain requirements to run. At the very least it needs a certain range of kernels and archs (and yes it's possible, although uncommon for a binary to support multiple archs). It may also require a network interface to bind a socket on a certain TCP port. It may require certain syscalls to be authorized. It may require the ability to create a tun/tap device. And so on. Docker's job is to offer a portable abstraction for these requirements, so that the container can list what it needs on the one hand, the host can list what it offers on the other, and docker can match them in the middle. If the requirements listed by a given container aren't met ("I need CAP_SYSADMIN on a 3.8 linux kernel and an ARM processor!") then docker returns an error and a clear error message. If they are met, the container is executed and must always be repeatable. TLDR: ABI requirements are just one kind of requirements. Docker can handle multiple requirements without breaking the repeatability of its execution environment.

deleted 13y ago

[deleted]

joeshaw 13y ago

Was the OS X binary built without cgo? I can't seem to access containers in private https registries: $ docker login https://registry.example.com 2014/02/05 14:36:20 Invalid Registry endpoint: Get https://registry.example.com/v1/_ping: x509: failed to load system roots and no roots provided The hostname in question has a valid SSL certificate. I encountered a similar problem in the past with Go built from homebrew[1][2]. Has anyone else seen this? [1] https://github.com/Homebrew/homebrew/pull/17758 https://github.com/Homebrew/homebrew/pull/17758 [2] https://code.google.com/p/go/issues/detail?id=4791 https://code.google.com/p/go/issues/detail?id=4791 Update: Filed a bug against docker, others are having the same issue. https://github.com/dotcloud/docker/issues/3946 https://github.com/dotcloud/docker/issues/3946

crosbymichael 13y ago
Thanks for the report. We are working on a fix ASAP and will post updates on https://github.com/dotcloud/docker/issues/3683 https://github.com/dotcloud/docker/issues/3683.
- jschorr 13y ago
  In the meantime, if you need to get it working right now, you can build your own binary as outlined here: http://blog.devtable.com/2014/01/using-docker-on-osx-with-private.html http://blog.devtable.com/2014/01/using-docker-on-osx-with-pr... We've confirmed the instructions still work with Docker 0.8 (make sure to change the checkout branch though :))

sbt 13y ago

I am interested in the BTRFS support in particular, it is clear that performance in FS is key. However, what I like the most about Docker is the ability to use layers and diff them. In effect, I want version control for images, because it allows me to not run extra provisioning tools for the images (just rely on simply Zookeeper stuff for app config). Whatever gives me 'vcs' for images in the most performant way, wins in my book.

jokoon 13y ago

can someone explain what this is ? and what's the purpose ?

jamtur01 13y ago
Try here: http://www.docker.io/learn_more/ http://www.docker.io/learn_more/
- jokoon 13y ago
  nope, I still don't get it. > Docker is an open-source engine that automates the deployment of any application as a lightweight, portable, self-sufficient container that will run virtually anywhere. Like an internet browser for executables ? I don't understand how can this be useful...
  - McGlockenshire 13y ago
    Containers are like virtual machines, only without the hardware emulation. See http://en.wikipedia.org/wiki/LXC http://en.wikipedia.org/wiki/LXC See http://docs.docker.io/en/latest/faq/#how-do-containers-compare-to-virtual-machines http://docs.docker.io/en/latest/faq/#how-do-containers-compa... Docker is a mechanism to bundle an application together inside a container (think VM instance) in a way that makes it easier to distribute.
  - ecnahc515 13y ago
    Think of it like this: Say you have a python/rails/nodejs/c++/whatever app. Sometimes getting all the dependencies on the system is cumbersome and hard to manage. This is true for both developers, and the people deploying these apps. I can't count how many times I've had a python app fail to build on a new box because of some C extension and forgetting to install a package on (centOS, ubuntu, debian, ect). Docker lets me do all of this once for my app, with a Docker image, and now when I want to deploy it, all the system needs is Docker installed. This means on my laptop, on our staging server, on our production server, all they need is Docker, and I will have the exact same environment in each place, and to deploy the app, its exactly the same. There's a ton of other bonuses like each container's dependencies and processes are isolated from each other. I also get a ton of the Docker features that allow containers to communicate with each other and setup service discovery between them (ex: your database container can now expose information to the app container using environmental variables). Tons of other good reasons too though, you should check it out.

stesch 13y ago

Is Docker a good way to bring more security on a server with a few different websites? Separating the sites from each other and run nginx as a proxy in front of them? What's the overhead?

zobzu 13y ago
its not as good as using separate vms, but has arguably lower overhead also, its not really docker doing that, its LXC. Docker is an API around it.
- nickstinemates 13y ago
  Docker is not an API around LXC. Any more than any application using a framework is a wrapper around said framework. However, the analogy I just made is only at a point in time. LXC is not the only Linux provider to manage cgroups/namespaces, it was just the most convenient at the time for the target audience. That is a fleeting position soon remedied. If you'd like to know more, I'd encourage you to get involved with Docker development.
  - zobzu 13y ago
    yeah, docket is an api around namespaces.. and thus lxc, right now. thanks for nothing really.
FireBeyond 13y ago
Yes, it is. It’s atop LXC (Linux Containers), which - oversimplified - is process namespacing, jails, etc. There’s near zero overhead, because there’s no virtualization.
midas007 13y ago
It depends. If the goal is to consolidate several boxes to a single VM, it does this. Be sure host-based (on the linux box) firewall rules are set and documented. If possible, set network-based firewall rules also (AWS security groups).
_wmd 13y ago

midas007 13y ago

It's confusing why btrfs support was prioritized ahead of zfs considering zfs' superior architecture and ops capabilities. Is docker (formerly dotcloud) going to start withholding capabilities as licensed features? Edit: prelim zfs driver work is here https://github.com/gurjeet/docker/tree/zfs_driver https://github.com/gurjeet/docker/tree/zfs_driver

tacticus 13y ago
What superior architecture does zfs have?
- rch 13y ago
  My understanding is that btrfs is still in catchup mode for the foreseeable future, but might eventually cover the distance. Has btrfs jumped ahead of zfs in ways I haven't heard about? Edit - this is my first search result: http://rudd-o.com/linux-and-free-software/ways-in-which-zfs-is-better-than-btrfs http://rudd-o.com/linux-and-free-software/ways-in-which-zfs-...
  - midas007 13y ago
    I'd be biased to agree since that's Manuel's blog, someone I used to work with. I've supported 24x7 and 9x5 ops where downtime was unacceptable. zfs makes it a whole lot easier to perform upgrades, know data and metadata are solid and send snapshots around.
    - rch 13y ago
      > ZFS uses atomic writes and barriers This about settles the question for me. Assuming that the implication that btrfs performs otherwise holds true.
      - midas007 13y ago
        Yeah, it depends on the use case. For home directories and large risk items like financial stuff, testing that barrier writes are happening is a good thing. You don't want a storm to knock out a DC to learn that the hw/sw fs stack was lying to you at some level.

brunoqc 13y ago

Anyone using Docker on 32-bits?

jamtur01 13y ago
Docker does not (yet) support 32-bit architectures.
- brunoqc 13y ago
  I think I read that you just have to disable a condition somewhere and build your own base image or something like that.

newman314 13y ago

IMO, it would be fantastic if there was something like Docker for Windows. Imagine being able to bundle up games in individual containers and easily being able to move them from machine to machine as you upgrade. Same thing applies for other Windows apps.

smtddr 13y ago
The developers of DRM for Windows games would probably make sure that doesn't work.
- jamtur01 13y ago
  Not if it's an option for installation. Or if they start shipping them like that. Steam springs to mind. :)
wslh 13y ago
Docker is to Unix almost as Microsoft App-V/VMware ThinApp/Symantec Workspace Virtualization products are to Windows. Not exactly the same but closer.
- newman314 13y ago
  Precisely, it's not the same. There still a whole bunch of fiddling going on. I'll stick to my statement. I want something Docker-like for Windows so that I can easily move things from one machine to another.
  - wslh 13y ago
    It seems like they are working on it: "Microsoft Corporation : Patent Issued for Extensible Application Virtualization Subsystems" http://www.4-traders.com/MICROSOFT-CORPORATION-4835/news/Microsoft-Corporation--Patent-Issued-for-Extensible-Application-Virtualization-Subsystems-17940954/ http://www.4-traders.com/MICROSOFT-CORPORATION-4835/news/Mic...

optymizer 13y ago

So what's the solution for 'root inside a docker container is root on the host'? We'd like to ship a set of utilities as a docker container, but unless the sysadmin gives everyone 'sudo' privileges on the server (unlikely and insecure), they can't run the container and its utilities. Any advice?

shykes 13y ago
Look for an update on this in 0.9 :) Future versions of the Docker API will natively support scoping. This means that each API client will see a different subset of the Docker engine depending on the credentials and origin of the connection. This will be implemented in combination with introspection, which allows any container to open a connection to the Docker engine which started it. When you combine scoping and introspection, you get really cool scenarios. For example, let's say your utility is called "dockermeister". Each individual user could deploy his own copy of dockermeister, in a separate container. Each dockermeister container would in turn connect to Docker (via introspection), destroy all existing containers, and create 10 fresh redis containers (for reasons unknown). Because each dockermeister container is scoped, it can only remove containers that are its children (ie that were created from the same container at an earlier time). So they cannot affect each other. Likewise, the 10 new redis containers will only be visible to that particular user, and not pollute the namespace of the other users. Of course scoping works at arbitrary depth levels... so you could have containers starting containers starting containers. Containers all the way down :)
kapilvt 13y ago
user namespaces will solve this. lxc in ubuntu 14.04 already supports docker in user lxc, ie. no root needed. add silly disclaimer.. yes docker has some notion of portable containers plugins, but it uses lxc atm, and the feature is in lxc upstream.

andrewcooke 13y ago

why is osx support necessary on the path to 1.0? i'd rather have a simple, small, 1.0 release i can trust than all these "bells and whistles".

shykes 13y ago
It's not necessary and we didn't go out of our way to get it. We just happened "for free" as a result of writing portable code, a clean client-server architecture and the appearance of the boot2docker project in the community.

morgante 13y ago

Awesome release, especially since I can now use Docker on OS X without having to boot up a full Ubuntu VM through Vagrant. If anyone else is using Boxen, I packaged up a quick Puppet module to get up and running with Docker on OS X: https://github.com/morgante/puppet-docker https://github.com/morgante/puppet-docker