3 ms·

I thought the same thing about Rabbit. PinLater's ACK system feels like a questionable reinvention of AMQP. I'm probably missing something. Rabbit's advertis

by bjt 12y ago

I thought the same thing about Rabbit. PinLater's ACK system feels like a questionable reinvention of AMQP. I'm probably missing something. Rabbit's advertised throughput doesn't reach 100,000/sec, but you should be able to shard your way around that. Or use Apache Kafka, which has the sharding built in.

fintler 12y ago
Just for comparison, LinkedIn is doing about 2.5 million messages per second (on average per day) with Apache Kafka. http://www.slideshare.net/ToddPalino/enterprise-kafka-kafka-as-a-service http://www.slideshare.net/ToddPalino/enterprise-kafka-kafka-...
- hatred 12y ago
  Precisely, this is the same query I had when I glanced down the post. It will be great if the author or someone from the team respond to the various trade-offs and why they did not go ahead with Kafka ?
  - omrvp 12y ago
    (I am the author of the blog post). I posted a reply below earlier this morning that hopefully addresses your question.
bigtones 12y ago
Looks like they needed to have built-in support for time based scheduled execution of jobs at a specific time in the future. Rabbit MQ does not have that (that I'm aware of) without doing some horrible hacks around one queue per message.
- kondro 12y ago
  ActiveMQ's had that for a couple of versions though. Another highly performant/available, atomic, consistent, durable and mature queueing server.
- cyounkins 12y ago
  Correct. Celery's implementation dequeues but doesn't acknowledge the task, holding it until the timer is reached.