4 ms·
I have found that using KillMode=mixed is very useful when running Airflow with Celery workers. It allows the system to shudown gracefully coupled with TimeoutS
by villasv 7y ago
I have found that using KillMode=mixed is very useful when running Airflow with Celery workers. It allows the system to shudown gracefully coupled with TimeoutStopSec, so the workers will stop receiving new jobs but will finish their current jobs before exiting which is nice for auto scaling or spot instances on AWS (coupled with EC2 lifecycle hooks).
- 1996 7y agoInteresting. Why do you care about graceful shutdown? Why not kill ASAP and restart? Do you have problems with load booming up sometimes?
- akramar 7y agoIn some cases we need to do an update-deploy-restart while a DAG is still running (not even the one being updated). Then several minutes or hours later the child processes raise a segfault and the jobs they were working on fail, requiring restarting any of those jobs. I imagine a graceful shutdown would allow the job to finish up and the DAG to continue with the remaining jobs.