2 ms·
I have been working on a Django application with two periodic background tasks to synchronize some data with some hardware, I was using Celery+Redis to handle t
by 1_player 11y ago
I have been working on a Django application with two periodic background tasks to synchronize some data with some hardware, I was using Celery+Redis to handle the schedulation. The solutions for running cron-like tasks on Python are not very mature, everybody seems to be using Celery.
Once deployed in production I spent countless evenings debugging why the tasks weren't running, why upon timeouts tasks weren't terminated, and why the task queue was getting bigger and bigger. 50% were configuration problems, due to the _bad_ and confusing documentation, 50% were due to introducing a sheer amount of complexity with Celery that I often spent my time reading through the source code to understand what was going on. And Python, with all its magic methods and abstractions, made that very hard.
One day I just rewrote everything with three Django management commands: two for the background tasks, one for the schedulation (with multiprocessing). I haven't heard from the client since.