5 ms·

RabbitMQ Internals

253 points by blopeur 11y ago

jvehent 11y ago
This is cool! RabbitMQ could really use better documentation. I had to implement permissions for MIG a while ago, and it tooks many trials and errors to get it right. https://github.com/mozilla/mig/blob/master/tools/create_rabbitmq_config.sh https://github.com/mozilla/mig/blob/master/tools/create_rabb...
- nitrogen 11y ago
  I found RabbitMQ's docs to be pretty useful myself, as long as you are familiar with regular expressions. I set up a system with fewer users and simpler permissions though. Out of curiosity, what did you find lacking?
  - jvehent 11y ago
    Use cases and examples are pretty scarce. The doc is accurate, but understand the doc for a newcomer to the tool is difficult.
    - dgritsko 11y ago
      I highly recommend "RabbitMQ in Action" (https://www.manning.com/books/rabbitmq-in-action https://www.manning.com/books/rabbitmq-in-action). It has been an invaluable reference in incorporating RabbitMQ into our production environment.
redwards510 11y ago
This will be helpful in understanding this framework I found yesterday called RestBus[1]. From their site: "RestBus is a high performance messaging library that enables you to consume ASP.NET Core, Web API and ServiceStack service endpoints via RabbitMQ". [1] http://restbus.org/ http://restbus.org/
elcct 11y ago
Oh that is so sweet. I was looking for something to read for bed time and this is perfect. Thanks!
blorgle 11y ago
"Finally, GM is not at all designed to cope with partial network partitions: if A is partitioned from B, then B can remove it from Mnesia, and that information can leak back to A via C. We currently don't handle this situation well; this is the biggest unsolved problem in RabbitMQ." Thankyou for acknowledging this! The mirroring is functionally not a very useful HA/horizontal scale feature until this is sorted. Mnesia has the same problem, so one has to imagine that the root cause is Mnesia. Running RabbitMQ in production for large scale OpenStack deployments, I have actually started to recommend a singleton RabbitMQ instance because the mirroring can cause all kinds of emergent behaviour during a network partition. OTOH, RabbitMQ does handle OpenStack at scale very well when there are no partitions, which is no mean feat, so kudos and thankyou.
- louthy 11y ago
  > Thankyou for acknowledging this! Indeed! We stopped using RabbitMQ for this reason. Actually, for a number of reasons (its slowness being another major one), but this was the biggy. As a fault tolerant system it doesn't tolerate faults very well at all; we just could not find a way of keeping it stable for long periods, so it had to go.
  - tokenizerrr 11y ago
    What do you use instead?
    - EGreg 11y ago
      ZeroMQ?
    - xordon 11y ago
      Kafka
    - louthy 11y ago
      I wrote my own built on Redis and the StackExchange.Redis client. Then created an actor-system layer on top of that. It's called LanguageExt.Process and is part of my LanguageExt project: https://github.com/louthy/language-ext https://github.com/louthy/language-ext

253 points by blopeur 11y ago

This is cool! RabbitMQ could really use better documentation. I had to implement permissions for MIG a while ago, and it tooks many trials and errors to get it right. https://github.com/mozilla/mig/blob/master/tools/create_rabbitmq_config.sh https://github.com/mozilla/mig/blob/master/tools/create_rabb...

nitrogen 11y ago
I found RabbitMQ's docs to be pretty useful myself, as long as you are familiar with regular expressions. I set up a system with fewer users and simpler permissions though. Out of curiosity, what did you find lacking?
- jvehent 11y ago
  Use cases and examples are pretty scarce. The doc is accurate, but understand the doc for a newcomer to the tool is difficult.
  - dgritsko 11y ago
    I highly recommend "RabbitMQ in Action" (https://www.manning.com/books/rabbitmq-in-action https://www.manning.com/books/rabbitmq-in-action). It has been an invaluable reference in incorporating RabbitMQ into our production environment.

redwards510 11y ago

This will be helpful in understanding this framework I found yesterday called RestBus[1]. From their site: "RestBus is a high performance messaging library that enables you to consume ASP.NET Core, Web API and ServiceStack service endpoints via RabbitMQ". [1] http://restbus.org/ http://restbus.org/

elcct 11y ago

Oh that is so sweet. I was looking for something to read for bed time and this is perfect. Thanks!

blorgle 11y ago

"Finally, GM is not at all designed to cope with partial network partitions: if A is partitioned from B, then B can remove it from Mnesia, and that information can leak back to A via C. We currently don't handle this situation well; this is the biggest unsolved problem in RabbitMQ." Thankyou for acknowledging this! The mirroring is functionally not a very useful HA/horizontal scale feature until this is sorted. Mnesia has the same problem, so one has to imagine that the root cause is Mnesia. Running RabbitMQ in production for large scale OpenStack deployments, I have actually started to recommend a singleton RabbitMQ instance because the mirroring can cause all kinds of emergent behaviour during a network partition. OTOH, RabbitMQ does handle OpenStack at scale very well when there are no partitions, which is no mean feat, so kudos and thankyou.

louthy 11y ago
> Thankyou for acknowledging this! Indeed! We stopped using RabbitMQ for this reason. Actually, for a number of reasons (its slowness being another major one), but this was the biggy. As a fault tolerant system it doesn't tolerate faults very well at all; we just could not find a way of keeping it stable for long periods, so it had to go.
- tokenizerrr 11y ago
  What do you use instead?
  - EGreg 11y ago
    ZeroMQ?
  - xordon 11y ago
    Kafka
  - louthy 11y ago
    I wrote my own built on Redis and the StackExchange.Redis client. Then created an actor-system layer on top of that. It's called LanguageExt.Process and is part of my LanguageExt project: https://github.com/louthy/language-ext https://github.com/louthy/language-ext