4 ms·

Would not this be an ideal Docker database?! But I could not find a good maintained Docker image in the hub: https://hub.docker.com/search/?isAutomated=0&isOffi

Would not this be an ideal Docker database?! But I could not find a good maintained Docker image in the hub: https://hub.docker.com/search/?isAutomated=0&isOfficial=0&page=1&pullCount=0&q=postgres+bdr&starCount=0 https://hub.docker.com/search/?isAutomated=0&isOfficial=0&pa...

cookiecaper 10y ago
Docker and database do not mix.
- thom_nic 10y ago
  Source? I have not seen strong argument against running postgres or other DBs in a container. Data directory should be mapped to a persistent volume, of course.
  - superuser2 10y ago
    Then why would you run it in a container? You're not encapsulating anything or making anything portable of the interesting part of the container is a directory on the host.
    - therealmarv 10y ago
      You can make the whole binary/running part interchangeable... think of OS updates, Postgres updates etc. just replace the container with a new one.
      - superuser2 10y ago
        >just Why is building, distributing, and cutting over to a new container easier than "sudo apt-get upgrade postgres"?
        therealmarv 10y ago
        Nobody said it's easier. But it's definitely more decoupled with a more well defined and reproducable binary image state which you can spin up and shut down in large scale easier. Nobody is saying that this is something you, your Postgres or most people need. Over 10+ million Docker hub pulls of the Postgres images saying there is a use case of using Postgres Docker images.
  - cookiecaper 10y ago
    Source is a basic understanding of containers and a basic understanding of databases. a) Docker and other container systems are still very young. They're missing a lot of badly-desired features and the community is still coalescing on best practices and safe approaches. It's certain there are complex and significant bugs. This is triply-true if you're using an orchestration framework over the top like Kubernetes. This isn't a reason why DBs are a bad fit in itself, but it's a reason why running production data in Docker is a bad idea at present. b) Docker is meant for stuff that's portable and can be isolated from its hardware. It's meant to make it easy to run many applications on one machine without the resource overhead of virtual machines. DBs are distinctly not well-suited to this environment. DBs require a good deal of tuning and hardware awareness to run properly. DBs want to run forever and keep frequently-used data in memory; often, restarting a DB server is a big deal because it makes the caches go cold and the server is slow until they warm back up. DBs don't take kindly to sharing a box with 30 other applications. On a VM at least you know you have a dedicated chunk of memory. Not the case on a Docker host. As I discuss in C, Docker is basically the antithesis of long-running; there are many ways to make your container suddenly disappear or stop. c) There are many non-obvious gotchas involved in Docker usage; it's a very non-intuitive interface. For example, using `docker attach` to connect to a running container will often place the user in control of the process. A simple ^C, which most sysadmins would interpret as "OK, I'm done with this log", closes the container's process and cleans up your container. That's just one example of a risk among many brought to you by the counter-intuitive and unfriendly Docker UX. Not the kind of fragility we want for something as mission-critical as a database system. Again, Docker is meant for processes that are consequence-free if they're cleaned up. Their UX is obviously designed that way too. You're supposed to run 8 containers from the same image and if one goes away the LB detects it and it's nbd. Databases don't work like that. d) Docker sometimes becomes a zombie and fails to respond to commands. You can't connect to any of your running containers. You can't issue start or stop commands. You can't get output from `docker ps`. This has happened to me on multiple occasions. Do we really want a production DB running in that context? e) You mentioned it, but storage. Even persistent volumes can be a PITA to configure properly with Docker, and if you fail to do so, you are looking down a blackhole backed by the slow AUFS virtual filesystem. By default, all data written to a Docker container goes into its own differenced image and when you stop the container, it "goes away" (though it can usually be recovered by calling up the specific container ID instead of restarting from the image). These AUFS volumes have a habit of consuming a lot of disk space on the host, which sometimes causes point d to occur if you get to 0. Volumes cannot be mapped at build time and must be defined at runtime. There are many bugs with data volumes, including data loss bugs and filesystem feature bugs. The long and short of it is that Docker is oriented toward applications that aren't close to the metal and can have their log and write output easily redirected to more durable systems. This fits the bill for many web apps (the application part, not the database parts). It absolutely does not fit the bill for long-running, close-to-the-metal, mandatory-firm-and-reliable-storage systems like databases. I use Docker to run local development databases (as well as applications). I wouldn't use it for any DB more important than that. Docker is a cool set of abstractions around jails, but it is not a universal solution, just as cloud isn't.
  - Kassandry 10y ago
    Josh Berkus, core PostgreSQL member and currently employed by Red Hat seems to think they go well together. Here's some of his work: https://www.youtube.com/watch?v=iruaCgeG7qs https://www.youtube.com/watch?v=iruaCgeG7qs https://github.com/jberkus/atomicdb https://github.com/jberkus/atomicdb
- Matt3o12_ 10y ago
  Care to elaborate?

cookiecaper 10y ago

Docker and database do not mix.

thom_nic 10y ago
Source? I have not seen strong argument against running postgres or other DBs in a container. Data directory should be mapped to a persistent volume, of course.
- superuser2 10y ago
  Then why would you run it in a container? You're not encapsulating anything or making anything portable of the interesting part of the container is a directory on the host.
  - therealmarv 10y ago
    You can make the whole binary/running part interchangeable... think of OS updates, Postgres updates etc. just replace the container with a new one.
    - superuser2 10y ago
      >just Why is building, distributing, and cutting over to a new container easier than "sudo apt-get upgrade postgres"?
      - therealmarv 10y ago
        Nobody said it's easier. But it's definitely more decoupled with a more well defined and reproducable binary image state which you can spin up and shut down in large scale easier. Nobody is saying that this is something you, your Postgres or most people need. Over 10+ million Docker hub pulls of the Postgres images saying there is a use case of using Postgres Docker images.
- cookiecaper 10y ago
  Source is a basic understanding of containers and a basic understanding of databases. a) Docker and other container systems are still very young. They're missing a lot of badly-desired features and the community is still coalescing on best practices and safe approaches. It's certain there are complex and significant bugs. This is triply-true if you're using an orchestration framework over the top like Kubernetes. This isn't a reason why DBs are a bad fit in itself, but it's a reason why running production data in Docker is a bad idea at present. b) Docker is meant for stuff that's portable and can be isolated from its hardware. It's meant to make it easy to run many applications on one machine without the resource overhead of virtual machines. DBs are distinctly not well-suited to this environment. DBs require a good deal of tuning and hardware awareness to run properly. DBs want to run forever and keep frequently-used data in memory; often, restarting a DB server is a big deal because it makes the caches go cold and the server is slow until they warm back up. DBs don't take kindly to sharing a box with 30 other applications. On a VM at least you know you have a dedicated chunk of memory. Not the case on a Docker host. As I discuss in C, Docker is basically the antithesis of long-running; there are many ways to make your container suddenly disappear or stop. c) There are many non-obvious gotchas involved in Docker usage; it's a very non-intuitive interface. For example, using `docker attach` to connect to a running container will often place the user in control of the process. A simple ^C, which most sysadmins would interpret as "OK, I'm done with this log", closes the container's process and cleans up your container. That's just one example of a risk among many brought to you by the counter-intuitive and unfriendly Docker UX. Not the kind of fragility we want for something as mission-critical as a database system. Again, Docker is meant for processes that are consequence-free if they're cleaned up. Their UX is obviously designed that way too. You're supposed to run 8 containers from the same image and if one goes away the LB detects it and it's nbd. Databases don't work like that. d) Docker sometimes becomes a zombie and fails to respond to commands. You can't connect to any of your running containers. You can't issue start or stop commands. You can't get output from `docker ps`. This has happened to me on multiple occasions. Do we really want a production DB running in that context? e) You mentioned it, but storage. Even persistent volumes can be a PITA to configure properly with Docker, and if you fail to do so, you are looking down a blackhole backed by the slow AUFS virtual filesystem. By default, all data written to a Docker container goes into its own differenced image and when you stop the container, it "goes away" (though it can usually be recovered by calling up the specific container ID instead of restarting from the image). These AUFS volumes have a habit of consuming a lot of disk space on the host, which sometimes causes point d to occur if you get to 0. Volumes cannot be mapped at build time and must be defined at runtime. There are many bugs with data volumes, including data loss bugs and filesystem feature bugs. The long and short of it is that Docker is oriented toward applications that aren't close to the metal and can have their log and write output easily redirected to more durable systems. This fits the bill for many web apps (the application part, not the database parts). It absolutely does not fit the bill for long-running, close-to-the-metal, mandatory-firm-and-reliable-storage systems like databases. I use Docker to run local development databases (as well as applications). I wouldn't use it for any DB more important than that. Docker is a cool set of abstractions around jails, but it is not a universal solution, just as cloud isn't.
- Kassandry 10y ago
  Josh Berkus, core PostgreSQL member and currently employed by Red Hat seems to think they go well together. Here's some of his work: https://www.youtube.com/watch?v=iruaCgeG7qs https://www.youtube.com/watch?v=iruaCgeG7qs https://github.com/jberkus/atomicdb https://github.com/jberkus/atomicdb
Matt3o12_ 10y ago
Care to elaborate?