4 ms·

> A sizable fraction will be concerned about 'container overhead' (and "scalability issues") when asked to move workloads to containers. They are usually not ab

> A sizable fraction will be concerned about 'container overhead' (and "scalability issues") when asked to move workloads to containers. They are usually not able to explain what the overhead would be, and what could potentially be causing it. For what it's worth, one of the biggest "containerization" recommendations is to not run your database (example: Postgres) in a container, correct? Due to I/O performance decrease?

wmf 7y ago
No. Docker volumes aka bind mounts have little or no overhead. You don't want to run a database in an ephemeral "cattle" container without some kind of HA because you'd lose data.
- MuffinFlavored 7y ago
  docker run --rm \ --name postgresql \ -e POSTGRES_PASSWORD=postgres \ -d \ -p 5432:5432 \ --cpuset-cpus="0-1" \ --cpus 2.0 \ -m=1024m \ --mount type=bind,source=$HOME/docker/volumes/postgres,target=/var/lib/postgresql/data \ postgres:12.2 native results on a digitalocean VM: $ pgbench -c 100 -j 2 -T 60 postgres starting vacuum...end. transaction type: <builtin: TPC-B (sort of)> scaling factor: 1 query mode: simple number of clients: 100 number of threads: 2 duration: 60 s number of transactions actually processed: 17660 latency average = 342.761 ms tps = 291.748482 (including connections establishing) tps = 291.791293 (excluding connections establishing) in a docker container: starting vacuum...end. transaction type: <builtin: TPC-B (sort of)> scaling factor: 1 query mode: simple number of clients: 100 number of threads: 2 duration: 60 s number of transactions actually processed: 13014 latency average = 466.928 ms tps = 214.165822 (including connections establishing) tps = 214.199201 (excluding connections establishing) 214tps in docker, 291tps outside of docker 26% decrease, with a bind mount on ubuntu 18.04
  - takeda 7y ago
    Docker introduces all kinds of performance issues. I for example noticed higher latency even with just pgbouncer. It was very visible when running integration tests, which normally took 15 minutes, but as a container it was 45min - 1h.
aprdm 7y ago
I think it is more because containers should be stateless and you cannot make a database stateless. We do run databases that do 100k`s ops/s in containers but we don't run them in kubernetes. We just mount the VM hard drive in it.
- Disposition 7y ago
  A DB container may and should be stateless, but when configured correctly the volumes specific to the storage engine are persistent. I've been running production databases in Docker since 2014 without any data loss, it makes a lot of system-level administrative work much easier. With a healthy understanding of how the individual storage engines commit to disk, upgrading, backing up, etc. can be done in parallel and without impact to a running production system thanks to the power of overlayfs.
deleted 7y ago
[deleted]
lazyant 7y ago
Funny, I looked into papers or articles about performance issues with containerized RDBS _binaries_ and didn't find anything relevant. Of course you want the data mounted outside the container so it's not ephemeral. I ran some casual tests using and found out there is a performance hit in using db binaries inside a Docker container due to Docker networking (different for different types of networking).
- takeda 7y ago
  I would be more concerned about writes going through additional fs layers and about abrupt termination of a container. You generally are trusting a database to keep your data safe, so those things will contribute to data loss. Remember the freakout about PostgreSQL not handling sync() correctly on Linux due to ambiguity in the man page? Having a networked filesystem + additional abstractions (like layers) etc only reduces data durability.
  - navaati 7y ago
    Except you never do that (having your DB directory be in the container rootfs). Nobody does, because then (beside the performance/reliability impact you mention) if the container goes (docker rm) the data goes. You're always gonna use a Volume for this kind of cases, be it K8s volume or Docker volume, and these, as neighbour message mentions, are just bind mounts (or actual mounts in certain cases), so no layers, no overlay, nothing of the sort.
    - MuffinFlavored 7y ago
      check out my response here: https://news.ycombinator.com/item?id=22809524 https://news.ycombinator.com/item?id=22809524 looks like docker with a bind mount has a 26% decrease in performance versus native

wmf 7y ago

No. Docker volumes aka bind mounts have little or no overhead. You don't want to run a database in an ephemeral "cattle" container without some kind of HA because you'd lose data.

MuffinFlavored 7y ago
docker run --rm \ --name postgresql \ -e POSTGRES_PASSWORD=postgres \ -d \ -p 5432:5432 \ --cpuset-cpus="0-1" \ --cpus 2.0 \ -m=1024m \ --mount type=bind,source=$HOME/docker/volumes/postgres,target=/var/lib/postgresql/data \ postgres:12.2 native results on a digitalocean VM: $ pgbench -c 100 -j 2 -T 60 postgres starting vacuum...end. transaction type: <builtin: TPC-B (sort of)> scaling factor: 1 query mode: simple number of clients: 100 number of threads: 2 duration: 60 s number of transactions actually processed: 17660 latency average = 342.761 ms tps = 291.748482 (including connections establishing) tps = 291.791293 (excluding connections establishing) in a docker container: starting vacuum...end. transaction type: <builtin: TPC-B (sort of)> scaling factor: 1 query mode: simple number of clients: 100 number of threads: 2 duration: 60 s number of transactions actually processed: 13014 latency average = 466.928 ms tps = 214.165822 (including connections establishing) tps = 214.199201 (excluding connections establishing) 214tps in docker, 291tps outside of docker 26% decrease, with a bind mount on ubuntu 18.04
- takeda 7y ago
  Docker introduces all kinds of performance issues. I for example noticed higher latency even with just pgbouncer. It was very visible when running integration tests, which normally took 15 minutes, but as a container it was 45min - 1h.

aprdm 7y ago

I think it is more because containers should be stateless and you cannot make a database stateless. We do run databases that do 100k`s ops/s in containers but we don't run them in kubernetes. We just mount the VM hard drive in it.

Disposition 7y ago
A DB container may and should be stateless, but when configured correctly the volumes specific to the storage engine are persistent. I've been running production databases in Docker since 2014 without any data loss, it makes a lot of system-level administrative work much easier. With a healthy understanding of how the individual storage engines commit to disk, upgrading, backing up, etc. can be done in parallel and without impact to a running production system thanks to the power of overlayfs.

deleted 7y ago

[deleted]

lazyant 7y ago

Funny, I looked into papers or articles about performance issues with containerized RDBS _binaries_ and didn't find anything relevant. Of course you want the data mounted outside the container so it's not ephemeral. I ran some casual tests using and found out there is a performance hit in using db binaries inside a Docker container due to Docker networking (different for different types of networking).

takeda 7y ago
I would be more concerned about writes going through additional fs layers and about abrupt termination of a container. You generally are trusting a database to keep your data safe, so those things will contribute to data loss. Remember the freakout about PostgreSQL not handling sync() correctly on Linux due to ambiguity in the man page? Having a networked filesystem + additional abstractions (like layers) etc only reduces data durability.
- navaati 7y ago
  Except you never do that (having your DB directory be in the container rootfs). Nobody does, because then (beside the performance/reliability impact you mention) if the container goes (docker rm) the data goes. You're always gonna use a Volume for this kind of cases, be it K8s volume or Docker volume, and these, as neighbour message mentions, are just bind mounts (or actual mounts in certain cases), so no layers, no overlay, nothing of the sort.
  - MuffinFlavored 7y ago
    check out my response here: https://news.ycombinator.com/item?id=22809524 https://news.ycombinator.com/item?id=22809524 looks like docker with a bind mount has a 26% decrease in performance versus native