4 ms·

I've never quite understood this pattern. What is the benefit of keeping data in a container when the main point of containers is that you can destroy them and

I've never quite understood this pattern. What is the benefit of keeping data in a container when the main point of containers is that you can destroy them and rebuild them at any time? It seems like the Docker ecosystem isn't built to support keeping state in a container. For me, the big thing is that once the data is in a container, it seems difficult to move around. How do you copy that container to a different system? In order to get to your data from the outside of a Docker container, do you need to use import/export? And even then, you just get a tar file? Why not use a volume mount to a local directory to begin with? Then you can still use the programs from within a container and still have easy access to your data when you are done. What am I missing? What is the advantage of using a data container?

fulafel 7y ago
Volume mounting host directories breaks the isolation and is a security risk. Data containers are more explicit about what happens in containers stays in containers. (I'm not saying data containers are super for everything, just that volume mounts from host are a lot worse)
- koffiezet 7y ago
  That would all be nice, if volumes would not be the exact same thing, where only the in-filesystem location is being managed by the docker daemon and not by the admin. In the end, both are just bind mounts to existing folders on the filesystem.
  - fulafel 7y ago
    Not necessarily, since the storage volume scenario doesn't have to be on the same host machine as the data you want to share to the container, you could eg share over a registry. But even in the same host case: 1) The most common use case of host volume mounts is usually to share your workspace to the container. That usually means mounting things from your project directory, maybe in your "src" or home directories, with the container, by default letting it write there too. This scenario is bad from both workflow hygiene and security POV, and there are clear UX advantages in the mental model of just using a more clearly bounded way of sharing data unidirectionally and in a explicit, controlled way. Plus you won't be asking other users to blindly trust your container with their host side files. 2) In many contexts (eg the Mac and Windows Docker apps) "docker" means containers running in virtual machines isolated from the host. 3) In Linux-native production setups with containers, the storage driver is using separate file systems, which offer isolation from many kinds of file descriptor and shared UID namespace based attacks from crossing over to other mounted filesystems (like the host OS and home directories).
mumblemumble 7y ago
I'm guessing the main advantage is that you can then use your Docker registry to distribute data sets. I remain skeptical that that's a good way to do this sort of thing, but it's certainly a way that would work.
koffiezet 7y ago
> I've never quite understood this pattern. It's a pattern that dates from when nobody really knew how to handle persistent storage with containers. The idea was that you could then move the data around using only the docker api I guess? I've seen it become 'best practice', and then disappear again in favour of "volumes" - which imho are just as useless, and failed to see the bigger picture. When running 'non-cloud-native' applications in docker-containers on snowflake servers, it - in my opinion - is indeed better to just map a local folder into the container, with docker-compose you can specify one relative to the docker-compose.yml file location, which makes this even more attractive. On k8s, you have storage providers/volumes/pvcs that attempt to address these needs, but I'm still a proponent of storing your data outside a k8s cluster as much as possible, and nothing that needs persistence should be stored in the containers, unless these are cluster-aware and handle their own clustering/replication and data storage.
CodesInChaos 7y ago
Mutable state and static assets/configuration are very different things. * For state you want to mount some kind of mutable storage that outlives the container. * Assets/config have very similar versioning and distribution requirements as code, so sharing a mechanism makes sense. If the assets are big and change less often than the application code, using a separate image helps keep the main application image small. (Though don't know enough about docker if the specific mechanism proposed in the linked article makes sense)

fulafel 7y ago

Volume mounting host directories breaks the isolation and is a security risk. Data containers are more explicit about what happens in containers stays in containers. (I'm not saying data containers are super for everything, just that volume mounts from host are a lot worse)

koffiezet 7y ago
That would all be nice, if volumes would not be the exact same thing, where only the in-filesystem location is being managed by the docker daemon and not by the admin. In the end, both are just bind mounts to existing folders on the filesystem.
- fulafel 7y ago
  Not necessarily, since the storage volume scenario doesn't have to be on the same host machine as the data you want to share to the container, you could eg share over a registry. But even in the same host case: 1) The most common use case of host volume mounts is usually to share your workspace to the container. That usually means mounting things from your project directory, maybe in your "src" or home directories, with the container, by default letting it write there too. This scenario is bad from both workflow hygiene and security POV, and there are clear UX advantages in the mental model of just using a more clearly bounded way of sharing data unidirectionally and in a explicit, controlled way. Plus you won't be asking other users to blindly trust your container with their host side files. 2) In many contexts (eg the Mac and Windows Docker apps) "docker" means containers running in virtual machines isolated from the host. 3) In Linux-native production setups with containers, the storage driver is using separate file systems, which offer isolation from many kinds of file descriptor and shared UID namespace based attacks from crossing over to other mounted filesystems (like the host OS and home directories).

mumblemumble 7y ago

I'm guessing the main advantage is that you can then use your Docker registry to distribute data sets. I remain skeptical that that's a good way to do this sort of thing, but it's certainly a way that would work.

koffiezet 7y ago

> I've never quite understood this pattern. It's a pattern that dates from when nobody really knew how to handle persistent storage with containers. The idea was that you could then move the data around using only the docker api I guess? I've seen it become 'best practice', and then disappear again in favour of "volumes" - which imho are just as useless, and failed to see the bigger picture. When running 'non-cloud-native' applications in docker-containers on snowflake servers, it - in my opinion - is indeed better to just map a local folder into the container, with docker-compose you can specify one relative to the docker-compose.yml file location, which makes this even more attractive. On k8s, you have storage providers/volumes/pvcs that attempt to address these needs, but I'm still a proponent of storing your data outside a k8s cluster as much as possible, and nothing that needs persistence should be stored in the containers, unless these are cluster-aware and handle their own clustering/replication and data storage.

CodesInChaos 7y ago

Mutable state and static assets/configuration are very different things. * For state you want to mount some kind of mutable storage that outlives the container. * Assets/config have very similar versioning and distribution requirements as code, so sharing a mechanism makes sense. If the assets are big and change less often than the application code, using a separate image helps keep the main application image small. (Though don't know enough about docker if the specific mechanism proposed in the linked article makes sense)