3 ms·

We decided not to use docker containers for our postgres DB in production; Volume mounts just don't make me sleep easy. We use an S3-backed private docker regis

We decided not to use docker containers for our postgres DB in production; Volume mounts just don't make me sleep easy. We use an S3-backed private docker registry to store our images/repositories.

cpuguy83 12y ago
I'd be interested to know what your concerns are with volumes.
vidarh 12y ago
What is your issue with volumes? Bind mounts have been battle tested over many years. I e.g. have production Gluster volumes bind-mounted into LXC containers that have been running uninterrupted for 5+ years.
- klochner 12y ago
  With docker specifically, I've had frustrations with the user permissions on the volume between {pg container, data container, host OS}, with extra trouble when osx/boot2docker is added to the stack. Also, docker doesn't add as much value for something like postgres that likely lives on it's own machine.
  - vidarh 12y ago
    > I've had frustrations with the user permissions on the volume between {pg container, data container, host OS} Then don't use data containers. I don't see much benefit from that either. The stuff we put on data volumes is stuff we want to manage the availability of very carefully, so I prefer more direct control. And so when I use volumes it's always bind mounts from the host. Some of them are local disk, some of them are network filesystems. We have some Gluster volumes that are exported from Docker containers that imports the raw storage via bind mounts from their respective hosts, for example, and then mounted on other hosts and bind-mounted into our other containers, just to make things convoluted - works great for high availability (I'm not recommending Gluster for Postgres, btw.; it "should" work with the right config, but I'd not dare without very, very extensive testing; nothing specific with Gluster, just generally terrified of databases on distributed filesystesms). > for something like postgres that likely lives on it's own machine. We usually colocate all our postgres instances with other stuff. There's generally a huge discrepancy between the most cost-effective amount of storage/iops, RAM and processing power if you're aiming for high density colocation, so it's far cheaper for us that way.
- peteridah 12y ago
  Nothing wrong with mounting volumes, but certainly not for production DB's, at least not at this point. I don't have much insight to share on this, besides my intuition based on several years working on HA systems. The clusterhq folks are working on this problem though, and I am following keenly.
  - shykes 12y ago
    Docker volumes are just bind-mounted directories. Of all the things you should worry about when running HA systems, bind-mounts is definitely not one of them. They are essentially overhead-free and completely stable. This is completely orthogonal to Docker - after setting up the bind-mounts it gets completely out of the way of the critical path. We have definitely used them at large scale in production for several years with no issues.
    - peteridah 12y ago
      Thanks shykes for chiming in on this. We use docker heavily for our applications in production (and all other envs), and would absolutely like to extend that to our Postgres DBs. Are there any examples you can point us to with a pattern for Postgres HA clusters with docker? Thanks.

We decided not to use docker containers for our postgres DB in production; Volume mounts just don't make me sleep easy. We use an S3-backed private docker registry to store our images/repositories.

cpuguy83 12y ago

I'd be interested to know what your concerns are with volumes.

vidarh 12y ago

What is your issue with volumes? Bind mounts have been battle tested over many years. I e.g. have production Gluster volumes bind-mounted into LXC containers that have been running uninterrupted for 5+ years.

klochner 12y ago
With docker specifically, I've had frustrations with the user permissions on the volume between {pg container, data container, host OS}, with extra trouble when osx/boot2docker is added to the stack. Also, docker doesn't add as much value for something like postgres that likely lives on it's own machine.
- vidarh 12y ago
  > I've had frustrations with the user permissions on the volume between {pg container, data container, host OS} Then don't use data containers. I don't see much benefit from that either. The stuff we put on data volumes is stuff we want to manage the availability of very carefully, so I prefer more direct control. And so when I use volumes it's always bind mounts from the host. Some of them are local disk, some of them are network filesystems. We have some Gluster volumes that are exported from Docker containers that imports the raw storage via bind mounts from their respective hosts, for example, and then mounted on other hosts and bind-mounted into our other containers, just to make things convoluted - works great for high availability (I'm not recommending Gluster for Postgres, btw.; it "should" work with the right config, but I'd not dare without very, very extensive testing; nothing specific with Gluster, just generally terrified of databases on distributed filesystesms). > for something like postgres that likely lives on it's own machine. We usually colocate all our postgres instances with other stuff. There's generally a huge discrepancy between the most cost-effective amount of storage/iops, RAM and processing power if you're aiming for high density colocation, so it's far cheaper for us that way.
peteridah 12y ago
Nothing wrong with mounting volumes, but certainly not for production DB's, at least not at this point. I don't have much insight to share on this, besides my intuition based on several years working on HA systems. The clusterhq folks are working on this problem though, and I am following keenly.
- shykes 12y ago
  Docker volumes are just bind-mounted directories. Of all the things you should worry about when running HA systems, bind-mounts is definitely not one of them. They are essentially overhead-free and completely stable. This is completely orthogonal to Docker - after setting up the bind-mounts it gets completely out of the way of the critical path. We have definitely used them at large scale in production for several years with no issues.
  - peteridah 12y ago
    Thanks shykes for chiming in on this. We use docker heavily for our applications in production (and all other envs), and would absolutely like to extend that to our Postgres DBs. Are there any examples you can point us to with a pattern for Postgres HA clusters with docker? Thanks.