> A recent example is WakaTime, who replaced Celery with a custom-built queue. This effort took one week to build and productionise, and consisted of just 1,264

> A recent example is WakaTime, who replaced Celery with a custom-built queue. This effort took one week to build and productionise, and consisted of just 1,264 lines of Python. What a load of BS. Wakatime doesn't even have tests, has been used by one person and has no error handler. It's 1k lines because it's alpha sotfware. Celery is not the most fun to play with, but it's not big for nothing. It supports many routers AND result stores, and you can mix an match. It has several policies regarding errors, has a beat daemon, provide an API for monitoring and managing (see: https://flower.readthedocs.io/en/latest/ https://flower.readthedocs.io/en/latest/ for UI), etc. Also, you can use celery in a very bare bone setup using only the FS for routing and storage: https://www.distributedpython.com/2018/07/03/simple-celery-setup/ https://www.distributedpython.com/2018/07/03/simple-celery-s... I understand you may want to avoid celery. I had bad experiences with it myself. Most projects can get away with rq, or even just multiprocessing.Pool. > Before embarking on the mission of creating a new task queue do survey the existing options, but make sure not to underestimate the hidden costs of using one, or the benefits that may come with writing one from scratch. Writting your own task queue is like creating your own csv parser. It seems like a very simple task that is barely a for loop with a split. And then you start hitting edge cases after edge cases. Basically you are coding an asynchronous, fault tolerant process manager including task serialization, communication protocol, priority queuing and lifecycle. That's not something you usually want to do. The cost of using one is unlikely higher that the cost of creating, maintaining and documenting one. Just go with a simple existing task queue if you need something simple. Pypi is full of those.