3 ms·

Evaluating persistent, replicated message queues

37 points by adamw1pl 12y ago

sargun 12y ago
What this article fails to talk about are primarily the failure scenarios of the queues under partitions. Does the queue become write-unavailable, read-unavailable, inconsistent, or otherwise during network partition? Can the queue remain partially available during network partition? These are the situations, in a scaling, production system that really matter.
- adamw1pl 12y ago
  These are very good questions, however for a different post. Here I was mainly interested in the throughput. Jepsen post provide some insights (into Mongo and Rabbit). With Hornet you can easily end up with two primaries. SQS is untestable in that respect :).
baq 12y ago
required reading for anyone using distributed systems: http://aphyr.com/tags/Jepsen http://aphyr.com/tags/Jepsen in particular: http://aphyr.com/posts/284-call-me-maybe-mongodb http://aphyr.com/posts/284-call-me-maybe-mongodb http://aphyr.com/posts/315-call-me-maybe-rabbitmq http://aphyr.com/posts/315-call-me-maybe-rabbitmq
- fasteo 12y ago
  tl;dr Distributed systems don´t work as advertised; not even close
- adamw1pl 12y ago
  Agreed, a must-read :). Bottom line is all queues can loose messages (we don't know much about SQS in fact).
whalesalad 12y ago
Would love to see Kafka represented here.
- adamw1pl 12y ago
  Yes, my bad, should be there :) Stay tuned, I'm out soon till end of next week, but will definitely add it.
srean 12y ago
I am not very familiar with messaging or message queues, so wanted to seek out some advice before spending time reinventing some well known wheel poorly. I would like to have a broker that sits between an application and several endpoint services to which the application makes requests via the broker. These endpoint services have rules, typically throttling rules (i) you cannot submit more than x number of request per second (ii) you cannot queue up more than y number of requests at our end (iii) you must put a delay of z seconds after we have responded to a request of yours, etc., etc. Is there a defacto standard open source software for this situation, that would take a config file describing the policies to follow for an end point. My requirement is pretty toy'ish, small to medium scale only. Just taking a look at Redis docs (am new to that too) it seems it will not be that hard to write out a simple solution. Would that be the way to go ? Or is there something out there that already handles this.
- adamw1pl 12y ago
  HornetQ has flow control (http://docs.jboss.org/hornetq/2.4.0.Final/docs/user-manual/html_single/index.html#flow-control http://docs.jboss.org/hornetq/2.4.0.Final/docs/user-manual/h...). Not sure about e.g. Rabbit. However I'm not aware of a standard solution in that space. Take a look at Reactive Streams (http://www.reactive-streams.org/ http://www.reactive-streams.org/) as well.
  - deleted 12y ago
    [deleted]
twic 12y ago
It would be nice to see the strongest of the various Apache contenders represented here too. Once upon a time, there was just ActiveMQ to worry about: http://activemq.apache.org/ http://activemq.apache.org/ It's long been very popular, and has all the flexibility and enterprisey goodness you could want, but has not had a reputation for speed. However, i believe there was a release a year or two ago that significantly improved performance. I vaguely remember this being based on an infusion of technology from the Qpid project, which itself has a broker: http://qpid.apache.org/components/java-broker/index.html http://qpid.apache.org/components/java-broker/index.html Or maybe that was what led to Apollo, which is the faster, more reliable successor to ActiveMQ, but which hasn't, er, succeeded ActiveMQ: http://activemq.apache.org/apollo/ http://activemq.apache.org/apollo/ Or maybe i just imagined the link to Qpid. I'm fairly hazy on how these bits of software relate, and i can't promise you there aren't more somewhere in the vast Apache mansion. Even figuring that out would be useful, never mind the benchmarks!
- adamw1pl 12y ago
  I'm also not sure how the above projects relate. As far as the scope of the blog goes, Apollo seems to be ruled out as it is not replicated. ActiveMQ does have replication, but they don't really sound convincing in the docs that it's first-class ;)

37 points by adamw1pl 12y ago

What this article fails to talk about are primarily the failure scenarios of the queues under partitions. Does the queue become write-unavailable, read-unavailable, inconsistent, or otherwise during network partition? Can the queue remain partially available during network partition? These are the situations, in a scaling, production system that really matter.

adamw1pl 12y ago
These are very good questions, however for a different post. Here I was mainly interested in the throughput. Jepsen post provide some insights (into Mongo and Rabbit). With Hornet you can easily end up with two primaries. SQS is untestable in that respect :).

baq 12y ago

required reading for anyone using distributed systems: http://aphyr.com/tags/Jepsen http://aphyr.com/tags/Jepsen in particular: http://aphyr.com/posts/284-call-me-maybe-mongodb http://aphyr.com/posts/284-call-me-maybe-mongodb http://aphyr.com/posts/315-call-me-maybe-rabbitmq http://aphyr.com/posts/315-call-me-maybe-rabbitmq

fasteo 12y ago
tl;dr Distributed systems don´t work as advertised; not even close
adamw1pl 12y ago
Agreed, a must-read :). Bottom line is all queues can loose messages (we don't know much about SQS in fact).

whalesalad 12y ago

Would love to see Kafka represented here.

adamw1pl 12y ago
Yes, my bad, should be there :) Stay tuned, I'm out soon till end of next week, but will definitely add it.

srean 12y ago

I am not very familiar with messaging or message queues, so wanted to seek out some advice before spending time reinventing some well known wheel poorly. I would like to have a broker that sits between an application and several endpoint services to which the application makes requests via the broker. These endpoint services have rules, typically throttling rules (i) you cannot submit more than x number of request per second (ii) you cannot queue up more than y number of requests at our end (iii) you must put a delay of z seconds after we have responded to a request of yours, etc., etc. Is there a defacto standard open source software for this situation, that would take a config file describing the policies to follow for an end point. My requirement is pretty toy'ish, small to medium scale only. Just taking a look at Redis docs (am new to that too) it seems it will not be that hard to write out a simple solution. Would that be the way to go ? Or is there something out there that already handles this.

adamw1pl 12y ago
HornetQ has flow control (http://docs.jboss.org/hornetq/2.4.0.Final/docs/user-manual/html_single/index.html#flow-control http://docs.jboss.org/hornetq/2.4.0.Final/docs/user-manual/h...). Not sure about e.g. Rabbit. However I'm not aware of a standard solution in that space. Take a look at Reactive Streams (http://www.reactive-streams.org/ http://www.reactive-streams.org/) as well.
- deleted 12y ago
  [deleted]

twic 12y ago

It would be nice to see the strongest of the various Apache contenders represented here too. Once upon a time, there was just ActiveMQ to worry about: http://activemq.apache.org/ http://activemq.apache.org/ It's long been very popular, and has all the flexibility and enterprisey goodness you could want, but has not had a reputation for speed. However, i believe there was a release a year or two ago that significantly improved performance. I vaguely remember this being based on an infusion of technology from the Qpid project, which itself has a broker: http://qpid.apache.org/components/java-broker/index.html http://qpid.apache.org/components/java-broker/index.html Or maybe that was what led to Apollo, which is the faster, more reliable successor to ActiveMQ, but which hasn't, er, succeeded ActiveMQ: http://activemq.apache.org/apollo/ http://activemq.apache.org/apollo/ Or maybe i just imagined the link to Qpid. I'm fairly hazy on how these bits of software relate, and i can't promise you there aren't more somewhere in the vast Apache mansion. Even figuring that out would be useful, never mind the benchmarks!

adamw1pl 12y ago
I'm also not sure how the above projects relate. As far as the scope of the blog goes, Apollo seems to be ruled out as it is not replicated. ActiveMQ does have replication, but they don't really sound convincing in the docs that it's first-class ;)