5 ms·
I do something like this in some projects of mine. RabbitMQ is completely happy to do this. On the other hand, literally every python client library would fall
by fake-name 4y ago
I do something like this in some projects of mine.
RabbitMQ is completely happy to do this. On the other hand, literally every python client library would fall over and/or wedge forever periodically. Celery in particular was a flaming pile of garbage (at least circa 2018).
I wound up vendoring the best of the bunch (amqpstorm, fyi), and adding a bunch of additional logic to handle all the various corner cases. That, plus a bunch of careful thought about how the queues are structured, and I've been pushing millions of messages a day without issue.
I wasn't able to get a lot of my changes upstream due to the extent I had to alter the assumptions the library made about control flow.
- OOPMan 4y agoDon't worry, Celery has always been and will always be a flaming pile of garbage
- xmcqdpt2 4y agoFor what it's worth, I found rq to be much nicer than celery but only used it in a relatively constrained fashion.
- justinclift 3y agorq doesn't interoperate with other languages though, so if you use it then your workers can only be written in Python. :(