5 ms·

I like this pattern of being able to restore snapshots in your development environment. For the uninitiated, docker actually has all these functions. For exampl

I like this pattern of being able to restore snapshots in your development environment. For the uninitiated, docker actually has all these functions. For example, if you create a container for postgres and mount a volume, you can create a snapshot of the volume for distribution for your team so they can mount/unmount anytime.

oauea 7y ago
> you can create a snapshot of the volume how? note that docker commit/export does not include volumes
- remmargorp64 7y ago
  This functionality for Docker was proposed by NetApp (https://netapp.io/2017/06/22/snapshots-clones-docker-volume-paradigm/ https://netapp.io/2017/06/22/snapshots-clones-docker-volume-...) There was also a github issue with numerous people voting for it (https://github.com/moby/moby/issues/33782 https://github.com/moby/moby/issues/33782) But for whatever reason, the docker team apparently decided not to build it. We still don't have a properly easy way to snapshot or share docker volumes. So I came up with my own method. Basically, you have to mount the volume and then tar the parts of it that you want snapshotted (typically your data directories). You can do this with any linux machine image. Here's an example (where "foo" is the docker container you want to snapshot): mkdir -p docker-volume-snapshots && docker run --rm --volumes-from foo -v $(shell pwd)/docker-volume-snapshots:/docker-volume-snapshots debian:stretch-slim tar -czvf /docker-volume-snapshots/snapshot.tar.gz /var/lib/my_data_folder Then, to restore the snapshot, here's what you do: docker run --rm --volumes-from mysql8.0 -v $(shell pwd)/docker-volume-snapshots:/docker-volume-snapshots:delegated debian:stretch-slim /bin/sh -c "cd / && tar -xvf /docker-volume-snapshots/snapshot.tar.gz" (Edit: Don't forget to start up/initialize and then pause/stop the running "foo" docker container before you run these commands to snapshot the volume)
  - oauea 7y ago
    Looks interesting but I doubt that'll be a consistent snapshit, right? I guess you could combine it with `docker pause`
    - remmargorp64 7y ago
      oh, yeah, I forgot to mention that I stop the running docker container before I do this
  - ericschrock 7y ago
    FYI, in the Titan project (https://titan-data.io https://titan-data.io), we solve this problem by running a privileges storage container (with ZFS on Linux) that then is able to export volumes into other containers, giving us ZFS snapshots and clones under the hood. This was inspired by the approach taken by dotmesh (https://dotmesh.com/ https://dotmesh.com/) but is definitely tailor-made to our use case. If someone was sufficiently motivated, I'm sure you could build a different CLI on top of our server and use it simply to manage docker volumes without the Titan trimmings (e.g. push and pull).
akhilcacharya 7y ago
I was under the impression container file systems make running a DB in a container a bad idea? Or is it just in performance critical situations?
- tehbeard 7y ago
  Doing so into the main container filesystem (which is usually an overlay/union stack) is probably bad for performance. But a bind mount (which is usually the recommended way to do it so that data survives the container lifespan) will avoid that and just leave you with whatever "problems" the host filesystem has re: database files.

oauea 7y ago

> you can create a snapshot of the volume how? note that docker commit/export does not include volumes

remmargorp64 7y ago
This functionality for Docker was proposed by NetApp (https://netapp.io/2017/06/22/snapshots-clones-docker-volume-paradigm/ https://netapp.io/2017/06/22/snapshots-clones-docker-volume-...) There was also a github issue with numerous people voting for it (https://github.com/moby/moby/issues/33782 https://github.com/moby/moby/issues/33782) But for whatever reason, the docker team apparently decided not to build it. We still don't have a properly easy way to snapshot or share docker volumes. So I came up with my own method. Basically, you have to mount the volume and then tar the parts of it that you want snapshotted (typically your data directories). You can do this with any linux machine image. Here's an example (where "foo" is the docker container you want to snapshot): mkdir -p docker-volume-snapshots && docker run --rm --volumes-from foo -v $(shell pwd)/docker-volume-snapshots:/docker-volume-snapshots debian:stretch-slim tar -czvf /docker-volume-snapshots/snapshot.tar.gz /var/lib/my_data_folder Then, to restore the snapshot, here's what you do: docker run --rm --volumes-from mysql8.0 -v $(shell pwd)/docker-volume-snapshots:/docker-volume-snapshots:delegated debian:stretch-slim /bin/sh -c "cd / && tar -xvf /docker-volume-snapshots/snapshot.tar.gz" (Edit: Don't forget to start up/initialize and then pause/stop the running "foo" docker container before you run these commands to snapshot the volume)
- oauea 7y ago
  Looks interesting but I doubt that'll be a consistent snapshit, right? I guess you could combine it with `docker pause`
  - remmargorp64 7y ago
    oh, yeah, I forgot to mention that I stop the running docker container before I do this
- ericschrock 7y ago
  FYI, in the Titan project (https://titan-data.io https://titan-data.io), we solve this problem by running a privileges storage container (with ZFS on Linux) that then is able to export volumes into other containers, giving us ZFS snapshots and clones under the hood. This was inspired by the approach taken by dotmesh (https://dotmesh.com/ https://dotmesh.com/) but is definitely tailor-made to our use case. If someone was sufficiently motivated, I'm sure you could build a different CLI on top of our server and use it simply to manage docker volumes without the Titan trimmings (e.g. push and pull).

akhilcacharya 7y ago

I was under the impression container file systems make running a DB in a container a bad idea? Or is it just in performance critical situations?

tehbeard 7y ago
Doing so into the main container filesystem (which is usually an overlay/union stack) is probably bad for performance. But a bind mount (which is usually the recommended way to do it so that data survives the container lifespan) will avoid that and just leave you with whatever "problems" the host filesystem has re: database files.