3 ms·
What about using Redis as a celery backend ? Redis has a pub sub mechanism which seems quite reliable, so no need to poll.
by mataug 12y ago
What about using Redis as a celery backend ? Redis has a pub sub mechanism which seems quite reliable, so no need to poll.
- denibertovic 12y agoRedis is still not an AMQP, but yes Redis's Pub/Sub works quite nicely. Out of all the brokers celery supports I'd recommend only RabbitMQ and Redis to people.
- mataug 12y agoYeah, I've been using redis with celery in production to perform lots of network io related tasks on a low end machine because a > Redis uses less memory b > Redis is easier to setup
- denibertovic 12y agoWith container solutions like Docker and prebuilt images the setup part is kinda eliminated. Although I don't remember having any special configuration issues with RabbitMQ as well, it just works TM. That's always nice right? :)
- bduerst 12y agoThat's been my experience with Redis + Celery as well vs. the other brokers.
- yen223 12y agoRedis works great as a results backend, but I'd still use RabbitMQ for the queue. RabbitMQ is designed to be a message queue, and it does a great job at it.
- TwistedWeasel 12y agoI used Redis for celery in production with great success for a year but then we started running some long running jobs that needed the ACKS_LATE setting and the Redis delivery timeout kept hurting us by resending the task to another worker. It's configurable but in the end we just switched to RabbitMQ. I found it quite painless to setup and migrate to.