55 ms·

I've never been sure if it's too much of a hack, but I've used GNU parallel in Docker containers as a quick and easy way of getting multiple processes running f

I've never been sure if it's too much of a hack, but I've used GNU parallel in Docker containers as a quick and easy way of getting multiple processes running for web applications. And with the `--halt now,done=1` option (that I think is relatively recent?) it means that if any of the parallel processes exit, parallel would exit itself, the whole container will shut down, and external orchestration would start another one if needed.

KronisLV 4y ago
I've used Supervisor pretty successfully for this as well: http://supervisord.org/ http://supervisord.org/ Example of installing it in a Debian/Ubuntu container during container build, here's an example Dockerfile: RUN apt-get update \ && apt-get -yq --no-upgrade install \ supervisor \ && apt-get clean \ && rm -rf /var/lib/apt/lists /var/cache/apt/* Then it's possible to create a configuration file, for example /etc/supervisord.conf, to specify what should run and how: [supervisord] nodaemon=true [program:php-fpm] command=/usr/sbin/php-fpm8.0 -c /etc/php/8.0/fpm/php-fpm.conf --nodaemonize stdout_logfile=/dev/stdout stdout_logfile_maxbytes=0 stderr_logfile=/dev/stderr stderr_logfile_maxbytes=0 [program:nginx] command=/usr/sbin/nginx stdout_logfile=/dev/stdout stdout_logfile_maxbytes=0 stderr_logfile=/dev/stderr stderr_logfile_maxbytes=0 And finally it can be run inside of the container entrypoint, along the lines of this in docker-entrypoint.sh: #!/bin/bash echo "Software versions..." nginx -V && supervisord --version echo "Running Supervisor..." supervisord --configuration=/etc/supervisord.conf Here's more information about the configuration file format, in case anyone is curious: http://supervisord.org/configuration.html http://supervisord.org/configuration.html It should be noted that this package will bring in some dependencies, though, which may or may not be okay, depending on how stringent you are about space usage and what's in your containers, example for a Ubuntu container: The following NEW packages will be installed: libexpat1 libmpdec3 libpython3-stdlib libpython3.10-minimal libpython3.10-stdlib libreadline8 libsqlite3-0 media-types python3 python3-minimal python3-pkg-resources python3.10 python3.10-minimal readline-common supervisor 0 upgraded, 15 newly installed, 0 to remove and 0 not upgraded. Need to get 6905 kB of archives. After this operation, 25.7 MB of additional disk space will be used. (just found the piece of software itself useful for this use case, figured I'd share my experiences) My problem is that it's not always immediately clear how software that would normally run as a systemd service could be launched in the foreground instead. It usually takes a bit of digging around.
- michalc 4y ago
  I have previously thought a bit about using something like Supervisor. And if I was running something a bit closer to the metal, with no other infrastructure to restart stuff, then I would be much more pro. But if inside Docker when something else already has the job of restarting things if they fall over, then it feels a bit over complicated in that there are multiple ways of doing the restarting. Plus, I think there is a touch more visibility - it's all just command line arguments to parallel: parallel --will-cite --line-buffer --jobs 2 --halt now,done=1 ::: \ "some_proc some args" \ "another_proc some more args"
- fbdab103 4y ago
  This is pretty crafty. I do not know supervisor well enough - if one of the services fail, can you engineer supervisor to also crash so that it would bubble up to the container infrastructure? My understanding is that standard supervisor would let the process die and/or restart the service.
  - KronisLV 4y ago
    Supervisor allows you to have event listeners (e.g. for processes quitting/crashing), so you can use those to achieve that and kill supervisor itself. Here's an example of people doing just that: https://gist.github.com/tomazzaman/63265dfab3a9a61781993212fa1057cb https://gist.github.com/tomazzaman/63265dfab3a9a61781993212f...
    - fbdab103 4y ago
      Neato. Do not have an immediate use case for this, but definitely something I will consider for the future.
vrnvu 4y ago
Cool tip thanks for sharing! I love letting process crash *when possible* on failures so the OS restart them for me versus trying to handle it manually at process level.

KronisLV 4y ago

I've used Supervisor pretty successfully for this as well: http://supervisord.org/ http://supervisord.org/ Example of installing it in a Debian/Ubuntu container during container build, here's an example Dockerfile: RUN apt-get update \ && apt-get -yq --no-upgrade install \ supervisor \ && apt-get clean \ && rm -rf /var/lib/apt/lists /var/cache/apt/* Then it's possible to create a configuration file, for example /etc/supervisord.conf, to specify what should run and how: [supervisord] nodaemon=true [program:php-fpm] command=/usr/sbin/php-fpm8.0 -c /etc/php/8.0/fpm/php-fpm.conf --nodaemonize stdout_logfile=/dev/stdout stdout_logfile_maxbytes=0 stderr_logfile=/dev/stderr stderr_logfile_maxbytes=0 [program:nginx] command=/usr/sbin/nginx stdout_logfile=/dev/stdout stdout_logfile_maxbytes=0 stderr_logfile=/dev/stderr stderr_logfile_maxbytes=0 And finally it can be run inside of the container entrypoint, along the lines of this in docker-entrypoint.sh: #!/bin/bash echo "Software versions..." nginx -V && supervisord --version echo "Running Supervisor..." supervisord --configuration=/etc/supervisord.conf Here's more information about the configuration file format, in case anyone is curious: http://supervisord.org/configuration.html http://supervisord.org/configuration.html It should be noted that this package will bring in some dependencies, though, which may or may not be okay, depending on how stringent you are about space usage and what's in your containers, example for a Ubuntu container: The following NEW packages will be installed: libexpat1 libmpdec3 libpython3-stdlib libpython3.10-minimal libpython3.10-stdlib libreadline8 libsqlite3-0 media-types python3 python3-minimal python3-pkg-resources python3.10 python3.10-minimal readline-common supervisor 0 upgraded, 15 newly installed, 0 to remove and 0 not upgraded. Need to get 6905 kB of archives. After this operation, 25.7 MB of additional disk space will be used. (just found the piece of software itself useful for this use case, figured I'd share my experiences) My problem is that it's not always immediately clear how software that would normally run as a systemd service could be launched in the foreground instead. It usually takes a bit of digging around.

michalc 4y ago
I have previously thought a bit about using something like Supervisor. And if I was running something a bit closer to the metal, with no other infrastructure to restart stuff, then I would be much more pro. But if inside Docker when something else already has the job of restarting things if they fall over, then it feels a bit over complicated in that there are multiple ways of doing the restarting. Plus, I think there is a touch more visibility - it's all just command line arguments to parallel: parallel --will-cite --line-buffer --jobs 2 --halt now,done=1 ::: \ "some_proc some args" \ "another_proc some more args"
fbdab103 4y ago
This is pretty crafty. I do not know supervisor well enough - if one of the services fail, can you engineer supervisor to also crash so that it would bubble up to the container infrastructure? My understanding is that standard supervisor would let the process die and/or restart the service.
- KronisLV 4y ago
  Supervisor allows you to have event listeners (e.g. for processes quitting/crashing), so you can use those to achieve that and kill supervisor itself. Here's an example of people doing just that: https://gist.github.com/tomazzaman/63265dfab3a9a61781993212fa1057cb https://gist.github.com/tomazzaman/63265dfab3a9a61781993212f...
  - fbdab103 4y ago
    Neato. Do not have an immediate use case for this, but definitely something I will consider for the future.

vrnvu 4y ago

Cool tip thanks for sharing! I love letting process crash *when possible* on failures so the OS restart them for me versus trying to handle it manually at process level.