You are right in our case redis failed spectacularly with celery queue when tasks in queue exceeded the memory of the machine. It took weeks to diagnose as erro

You are right in our case redis failed spectacularly with celery queue when tasks in queue exceeded the memory of the machine. It took weeks to diagnose as error message were inconsistent and setting up a Message Queue cluster with redis is a nightmare. Went back again to tried and tested rabbitMQ and it works so well. Also adding new nodes and removing nodes so easy, just used ansible to setup Erlang cookie and connect the node (thanks to OTP and BEAM). The best part is for important task queue for which we cannot accept failure we built a mechanism for fault tolerance. When you work with high availability and fault tolerant queue rabbitMQ is so good, it can recover from hardware or even VM failures, can’t say the same for redis which was a nightmare even with redis cluster.