3 ms·

I have to say that I have never really had a positive impression of zeromq. I feel like it is particularly unsuccessful and that seems to be for some pretty go

I have to say that I have never really had a positive impression of zeromq. I feel like it is particularly unsuccessful and that seems to be for some pretty good reasons. Zeromq is a bag of tools and it's not really clear to me how a novice would put them together safely. This is important for a message system because everyone has to start somewhere and it really shouldn't be "step 1: build your broker with our subroutines". Is that broker really going to be any good? Secondly, for many years zeromq was a kind of monty python style 'judean people's front vs people's front of judea' community. They split from the AMQP standardisation process I agree had clearly gone wildly wrong and then...split and then...split again. Right as far as I know people doing messaging are using a) RabbitMQ if they're deploying it themselves b) AWS of GCP's thing or c) some proprietary thing which has no name recognition among the public (who here has heard of AMPS?) d) Kafka. No one is using zeromq as far as I know! Look forward to anyone sharing their positive experiences using zeromq on big projects eg in banks.

rcxdude 6y ago
I agree. The basic interface design seems really neat but it's so easy to run into horrible problems with it. For one it doesn't really deal with errors very well: once you've thrown a message into it there's basically no mechanism to know if something goes wrong with it (and it certainly used to just straight up assert in many, many error cases, which sure made it hard to use the 'reliable central broker' mechanism they advocate in the docs). The implementation is also really brittle: it's basically trivial to cause all kinds of misbehaviour if the other end does not act as expected. The problem is there isn't really many good alternatives that are actually similar. Other 'message queues' are basically big databases, not a library. I feel there's room for something with the same philosophy but better execution.
filmor 6y ago
ZeroMQ is used heavily in the backend of Jupyter and allows for a huge variety of non-Python kernels. It's brokerless, which sets it apart from all the other things you listed here.
JonathonW 6y ago
As someone who's written more than my fair share of messaging code using a bunch of different middleware and frameworks, I've never really understood where ZeroMQ is supposed to fit. It's high-level enough that you have to learn something completely different than plain sockets (and have more stuff to debug when it breaks), while also being low-level enough that it can't provide many of the features I'd expect out of a message-oriented middleware-- no message broker, no anonymous pub-sub, no reliable transport, etc. unless you go and build them yourself (which they provide documentation and patterns for, but they're not things you get for free like you would with other frameworks). "Bag of tools" is about right. Where RabbitMQ/AMQP or MQTT or DDS can, depending on exactly what I want to do, provide me an end-to-end solution where, with a bit of configuration and network management, throw in a message and expect it to go where I want it to, ZeroMQ hands me a pile of pieces with which I can, presumably, build a thing that will do the same thing. That's not something I've ever found that I need-- either one of the premade solutions fills my needs, or I go and write something myself and skip ZeroMQ.
stock_toaster 6y ago
MQTT is also very big in mobile and IoT messaging. Perhaps as an (e) option in your list.
lllr_finger 6y ago
The only times I've ever reached for zeromq are when I needed sockets and didn't want to deal with the hassle of programming sockets - it worked great then. I'd happily use it as the glue between two processes that I control, but would grab one of the other tech you mentioned for anything exposed on the boundaries of my app.
ocschwar 6y ago
In 2011 I was a novice and got that bag of tools dropped in my lap, and I did struggle with how to put them together, but safely was not. problem. It was considerably better for a novice to play with those tools than to have to do it at the socket level.
ComputerGuru 6y ago
Forgive me as you seem to be aware of many of the details of zmq’s history but is there a chance you’re missing something? You are comparing a thin, semantic-enforcing layer just above TCP to actual message queues/servers featuring persistence to disk, etc. We’ve used ZeroMQ in production as an alternative to HTTP message passing and it was great for that. We would build something like RabbitMQ atop of ZMQ, the only similarity between the two being the name, really.
- calpaterson 6y ago
  I could well be missing something! I get that zeromq is not quite equivalent to rabbit or Google pubsub but you do presumably still want the same kinds of patterns: pubsub, queues and maybe a bit of routing? So it is an alternative to (eg) Google pubsub and I think can be compared with it. Is there any case where it would be used for a different /purpose/ than the others?
- hkon 6y ago
  Why would you build something similar to RabbitMQ ontop of ZeroMQ instead of just using Rabbit?
  - OJFord 6y ago
    I believe GP means that ZMQ could be an implementation detail of RMQ (itself). As in, in building your RabbitMQ competitor HareMQ, you might build on top of HareMQ. You can't just use HareMQ because you haven't built it yet.
    - ComputerGuru 6y ago
      More to the point, while you could build a theoretical HareMQ atop of ZMQ, you can also build a hundred different products/services that aren’t a RabbitMQ alternative out of it, too. Apples and oranges.

rcxdude 6y ago

I agree. The basic interface design seems really neat but it's so easy to run into horrible problems with it. For one it doesn't really deal with errors very well: once you've thrown a message into it there's basically no mechanism to know if something goes wrong with it (and it certainly used to just straight up assert in many, many error cases, which sure made it hard to use the 'reliable central broker' mechanism they advocate in the docs). The implementation is also really brittle: it's basically trivial to cause all kinds of misbehaviour if the other end does not act as expected. The problem is there isn't really many good alternatives that are actually similar. Other 'message queues' are basically big databases, not a library. I feel there's room for something with the same philosophy but better execution.

filmor 6y ago

ZeroMQ is used heavily in the backend of Jupyter and allows for a huge variety of non-Python kernels. It's brokerless, which sets it apart from all the other things you listed here.

JonathonW 6y ago

As someone who's written more than my fair share of messaging code using a bunch of different middleware and frameworks, I've never really understood where ZeroMQ is supposed to fit. It's high-level enough that you have to learn something completely different than plain sockets (and have more stuff to debug when it breaks), while also being low-level enough that it can't provide many of the features I'd expect out of a message-oriented middleware-- no message broker, no anonymous pub-sub, no reliable transport, etc. unless you go and build them yourself (which they provide documentation and patterns for, but they're not things you get for free like you would with other frameworks). "Bag of tools" is about right. Where RabbitMQ/AMQP or MQTT or DDS can, depending on exactly what I want to do, provide me an end-to-end solution where, with a bit of configuration and network management, throw in a message and expect it to go where I want it to, ZeroMQ hands me a pile of pieces with which I can, presumably, build a thing that will do the same thing. That's not something I've ever found that I need-- either one of the premade solutions fills my needs, or I go and write something myself and skip ZeroMQ.

stock_toaster 6y ago

MQTT is also very big in mobile and IoT messaging. Perhaps as an (e) option in your list.

lllr_finger 6y ago

The only times I've ever reached for zeromq are when I needed sockets and didn't want to deal with the hassle of programming sockets - it worked great then. I'd happily use it as the glue between two processes that I control, but would grab one of the other tech you mentioned for anything exposed on the boundaries of my app.

ocschwar 6y ago

In 2011 I was a novice and got that bag of tools dropped in my lap, and I did struggle with how to put them together, but safely was not. problem. It was considerably better for a novice to play with those tools than to have to do it at the socket level.

ComputerGuru 6y ago

Forgive me as you seem to be aware of many of the details of zmq’s history but is there a chance you’re missing something? You are comparing a thin, semantic-enforcing layer just above TCP to actual message queues/servers featuring persistence to disk, etc. We’ve used ZeroMQ in production as an alternative to HTTP message passing and it was great for that. We would build something like RabbitMQ atop of ZMQ, the only similarity between the two being the name, really.

calpaterson 6y ago
I could well be missing something! I get that zeromq is not quite equivalent to rabbit or Google pubsub but you do presumably still want the same kinds of patterns: pubsub, queues and maybe a bit of routing? So it is an alternative to (eg) Google pubsub and I think can be compared with it. Is there any case where it would be used for a different /purpose/ than the others?
hkon 6y ago
Why would you build something similar to RabbitMQ ontop of ZeroMQ instead of just using Rabbit?
- OJFord 6y ago
  I believe GP means that ZMQ could be an implementation detail of RMQ (itself). As in, in building your RabbitMQ competitor HareMQ, you might build on top of HareMQ. You can't just use HareMQ because you haven't built it yet.
  - ComputerGuru 6y ago
    More to the point, while you could build a theoretical HareMQ atop of ZMQ, you can also build a hundred different products/services that aren’t a RabbitMQ alternative out of it, too. Apples and oranges.