4 ms·
I'm actually a fan of rabbitMQ generally, but I'm not sure I'd put it in the category of notoriously reliable software. It can throw away acknowledged writes (a
by darkr 10y ago
I'm actually a fan of rabbitMQ generally, but I'm not sure I'd put it in the category of notoriously reliable software. It can throw away acknowledged writes (afaik https://aphyr.com/posts/315-jepsen-rabbitmq https://aphyr.com/posts/315-jepsen-rabbitmq has still not been fully fixed).
50k messages/day is pretty low throughout and under that load it should plod through just fine.
Under much higher loads than that (10's - 100's millions of messages day) I've seen it die in a pretty ungraceful manner. At that scale, Kafka is a good candidate.
- busterarm 10y agoUnder those high loads I've seen every single MQ fall over like a house of cards. At lighter loads you can get the job done with other tools that scale but with a little bit more pain. Leaves me super skeptical about them on the whole.
- cespare 10y agoWe push on the order of ~a billion messages / day into one of our Kafka clusters without having too much scaling difficulty.
- jstanier 10y agoIndeed. We have completely replaced all queues with Kafka. In fact, I'm not sure what it would take for us to even try anything else at this point.
- AznHisoka 10y agoWe process at least 30 million messages a day with sidekiq and had to start sharding message queues for different purposes. At some point it's not worth trying to optimize the messaging.