4 ms·
We solved the Celery + long running + code reloading problem by having each push of new code be associated with a new Celery queue. On push of new code, start t
by polpo 11y ago
We solved the Celery + long running + code reloading problem by having each push of new code be associated with a new Celery queue. On push of new code, start the new queue, and SIGTERM the old one, which will wait for any long-running tasks to finish before exiting.
- deleted 11y ago[deleted]
- ThePhysicist 11y agoHm that's an interesting idea, thanks for sharing this! How exactly do you perform the restart of the Celery service and where are your queues configured? I guess you don't specify them in /etc/defaults/celeryd?
- polpo 11y agoCorrect, we didn't use /etc/defaults/celeryd, or use a standard init/systemd/upstart/etc-based way of starting celeryd. Instead we used daemontools to add and remove celeryd processes with their own configuration. It took a bit of work, but ultimately let us do code updates with zero downtime.