5 ms·

Ever since I first read about docker I wondered: How are you dealing with application data and updates of containers? Let's say you package everything you need

by pilif 13y ago

Ever since I first read about docker I wondered: How are you dealing with application data and updates of containers? Let's say you package everything you need for your web application together as one container: Web Server, Database-Server and Application Server: Where would the data go that the users of that web applications generate? In the container? That would means that the data goes away as the container gets updated (if you just replace the old container with a new one). Or do you just never replace the whole container but just update what's inside? Or would you mount a device of the host? Does the container get access to device nodes of the parent? I would assume not. Or can you provide a container with something it can mount? Or are you stuck with some network based storage solution? That would rule out running databases in containers once the load raises. Yes. I could just read up on all of this, but I have a feeling that other people have the same question, so by asking here, I might help them too.

StavrosK 13y ago
You can mount a shared folder on the host and put things there (this is recent, AFAIK).
Joeboy 13y ago
I think this issue somewhat compromises Dockerfiles as a means of distributing applications. With a VM, you can supply something in a minimally configured, plug-and-play state, and then leave it to the user to set it up. With Docker, it's a little bit too painful to configure things after running an image. I'm not sure what the answer to this is.
- vidarh 13y ago
  Which is it: Minimally configured, or plug-and-play? Either you'll be setting up your containers to prepare you to use something like Chef or Puppet against the, or you'll be setting up your containers to be fully configured apart from connection details to other components. That last bit is down to service orchestration, which is outside the scope of Docker as far as I understand it. You can "roll your own" easily enough: Write a "/usr/local/bin/add-relation" script that takes a function and a list of arguments to configure that function. Write a script that reads a config file that defines the relationships between container types (e.g. your MySQL container and your Web container), and triggers execution of those add-relation scripts. E.g. your Web container gets called with "add-relation Mysql --ip [foo] --username [bar] --password [baz]" and the version of the script in that container knows how to add users to MySQL. That is very roughly the approach that Juju (Ubuntu's orchestration system) takes. The point is to split all configuration details into thress classes: Those that can be set statically on build. Those that can be decided on first boot (e.g. re-generate a ssh-host-key, get IP address etc.). And finally those that needs to be set dynamically based on what other containers are spun up. The latter has no place in the Dockerfiles or images, but scripts or tools to set these config details based on input from some external source, does. It may sound painful, but only if you do it to a handful of vm's that rarely change. The moment you need to manage dozens, or need to spin vm's up and down regularly, taking the effort to set up proper orchestration, whichever method you prefers, wins hands down to having a user set it up (I can guarantee with near 100% certainty that said user will not set them up different VMs consistently).
derefr 13y ago
Think of it like an EC2 instance with ephemeral storage. You need an external storage service provided by the host (like EBS and S3 are for EC2), that the instances either talk to over the network (like S3), or which the host mounts into the container (like EBS.) Both are possible options with the current Docker runtime.
- pilif 13y ago
  Yes. And we all know how well EBS works for consistent database performance. That's why I said "... would rule out running databases in containers ..."
  - vidarh 13y ago
    That is entirely irrelevant - it's an implementation detail of AWS. The volume could just as well be bind-mounted from the same local host.
    - pilif 13y ago
      That's what I was asking in my first question: Do you get access to a raw device node of the host OS?
      - yebyen 13y ago
        No, I don't think so. This is prevented by kernel namespaces and default privileges of LXC in Docker. That doesn't mean you can't bind mount a directory from your host into your container though; there's also the concept of volumes.
      - vidarh 13y ago
        I don't think so, but bind-mounts makes that irrelevant for most uses. If you need bare-metal device access, then you have very specialised needs that apply to a very tiny fringe set of users (as an example, we can reach 1GB/s reads from our SSD RAID arrays on some of our containerised database servers without resorting to raw device access).

by pilif 13y ago

StavrosK 13y ago

You can mount a shared folder on the host and put things there (this is recent, AFAIK).

Joeboy 13y ago

I think this issue somewhat compromises Dockerfiles as a means of distributing applications. With a VM, you can supply something in a minimally configured, plug-and-play state, and then leave it to the user to set it up. With Docker, it's a little bit too painful to configure things after running an image. I'm not sure what the answer to this is.

vidarh 13y ago
Which is it: Minimally configured, or plug-and-play? Either you'll be setting up your containers to prepare you to use something like Chef or Puppet against the, or you'll be setting up your containers to be fully configured apart from connection details to other components. That last bit is down to service orchestration, which is outside the scope of Docker as far as I understand it. You can "roll your own" easily enough: Write a "/usr/local/bin/add-relation" script that takes a function and a list of arguments to configure that function. Write a script that reads a config file that defines the relationships between container types (e.g. your MySQL container and your Web container), and triggers execution of those add-relation scripts. E.g. your Web container gets called with "add-relation Mysql --ip [foo] --username [bar] --password [baz]" and the version of the script in that container knows how to add users to MySQL. That is very roughly the approach that Juju (Ubuntu's orchestration system) takes. The point is to split all configuration details into thress classes: Those that can be set statically on build. Those that can be decided on first boot (e.g. re-generate a ssh-host-key, get IP address etc.). And finally those that needs to be set dynamically based on what other containers are spun up. The latter has no place in the Dockerfiles or images, but scripts or tools to set these config details based on input from some external source, does. It may sound painful, but only if you do it to a handful of vm's that rarely change. The moment you need to manage dozens, or need to spin vm's up and down regularly, taking the effort to set up proper orchestration, whichever method you prefers, wins hands down to having a user set it up (I can guarantee with near 100% certainty that said user will not set them up different VMs consistently).

derefr 13y ago

Think of it like an EC2 instance with ephemeral storage. You need an external storage service provided by the host (like EBS and S3 are for EC2), that the instances either talk to over the network (like S3), or which the host mounts into the container (like EBS.) Both are possible options with the current Docker runtime.

pilif 13y ago
Yes. And we all know how well EBS works for consistent database performance. That's why I said "... would rule out running databases in containers ..."
- vidarh 13y ago
  That is entirely irrelevant - it's an implementation detail of AWS. The volume could just as well be bind-mounted from the same local host.
  - pilif 13y ago
    That's what I was asking in my first question: Do you get access to a raw device node of the host OS?
    - yebyen 13y ago
      No, I don't think so. This is prevented by kernel namespaces and default privileges of LXC in Docker. That doesn't mean you can't bind mount a directory from your host into your container though; there's also the concept of volumes.
    - vidarh 13y ago
      I don't think so, but bind-mounts makes that irrelevant for most uses. If you need bare-metal device access, then you have very specialised needs that apply to a very tiny fringe set of users (as an example, we can reach 1GB/s reads from our SSD RAID arrays on some of our containerised database servers without resorting to raw device access).