4 ms·

My general experience with 0mq in the past (with a past team) has not been great. While a nice abstraction library over sockets, sending it the wrong parameter

My general experience with 0mq in the past (with a past team) has not been great. While a nice abstraction library over sockets, sending it the wrong parameter is quite likely to crash in painful ways in rare instances. Maybe it's gotten better over the past year. Yes, it's fast. But if you want a message bus and are going to want for advanced features, I'd recommend a proper message bus. It's going to abstract away some of the socket details, but you can still get bitten pretty easily - again, may have gotten better in the last several months, I don't know. Be careful also with distros carrying older versions, you are probably wanting to make sure you get the latest than what your distro may contain for the same reasons.

PieterH 11y ago
The original codebase used asserts rather poorly, for error handling rather than internal sanity checks. Four years ago we had fixed most of these problems, and had a "futzing" project to try to crash the library. Over time we've removed the edge cases... the codebase did take a long time to become properly stable, one of the downsides of lots of new code written rapidly, and one reason we insist on small gradual patches these days. (There have been _very_ few new errors introduced since we switched to our decentralized contribution process in early 2011.)
- mpdehaan2 11y ago
  Good to know. For what it's worth, the majority of my bad experiences were in 2013 to late 2014, and usually through the python bindings. Usually calling the functions in differnt ways was able to dodge the problem. It's not always quite obvious what caused it to crash, but it was sort of "if it hurts when you do that" sort of thing. Often these types of errors happened only under heavy traffic and were hard to replicate except under stress or lots of repeated calls. If not using the python bindings, it might be better off for most people.
- lobster_johnson 11y ago
  Has the problem with req/rep over lossy connections been solved (http://lucumr.pocoo.org/2012/6/26/disconnects-are-good-for-you/ http://lucumr.pocoo.org/2012/6/26/disconnects-are-good-for-y...)? My understanding was that you couldn't detect disconnects because ZMQ doesn't expose the internal state.
tomphoolery 11y ago
I've also worked with ZMQ in the past, and our experiences were a little better because we didn't use it as a message bus, but rather as a transport medium between modules in our proprietary video decoder software. It basically took away a lot of the boilerplate code you have to write around socket programming, which was really nice. If you're looking for an AMQP-compliant message bus that can be federated and clustered, RabbitMQ is pretty damn amazing.
- lobster_johnson 11y ago
  RabbitMQ is only amazing if your cluster is 100% available, you have no soft lockups, no high load, etc. Its cluster failure handling is absolute crap. In our stack it's the buggiest, most unreliable piece of software we are running, and it's falling over (losing cluster state, hanging, crashing, etc) on an almost daily basis. We used to run 8 nodes, and it was a nightmare recovering from partitions. Now we've reduced it to 2 per cluster, and it's still a nightmare.

PieterH 11y ago

The original codebase used asserts rather poorly, for error handling rather than internal sanity checks. Four years ago we had fixed most of these problems, and had a "futzing" project to try to crash the library. Over time we've removed the edge cases... the codebase did take a long time to become properly stable, one of the downsides of lots of new code written rapidly, and one reason we insist on small gradual patches these days. (There have been _very_ few new errors introduced since we switched to our decentralized contribution process in early 2011.)

mpdehaan2 11y ago
Good to know. For what it's worth, the majority of my bad experiences were in 2013 to late 2014, and usually through the python bindings. Usually calling the functions in differnt ways was able to dodge the problem. It's not always quite obvious what caused it to crash, but it was sort of "if it hurts when you do that" sort of thing. Often these types of errors happened only under heavy traffic and were hard to replicate except under stress or lots of repeated calls. If not using the python bindings, it might be better off for most people.
lobster_johnson 11y ago
Has the problem with req/rep over lossy connections been solved (http://lucumr.pocoo.org/2012/6/26/disconnects-are-good-for-you/ http://lucumr.pocoo.org/2012/6/26/disconnects-are-good-for-y...)? My understanding was that you couldn't detect disconnects because ZMQ doesn't expose the internal state.

tomphoolery 11y ago

I've also worked with ZMQ in the past, and our experiences were a little better because we didn't use it as a message bus, but rather as a transport medium between modules in our proprietary video decoder software. It basically took away a lot of the boilerplate code you have to write around socket programming, which was really nice. If you're looking for an AMQP-compliant message bus that can be federated and clustered, RabbitMQ is pretty damn amazing.

lobster_johnson 11y ago
RabbitMQ is only amazing if your cluster is 100% available, you have no soft lockups, no high load, etc. Its cluster failure handling is absolute crap. In our stack it's the buggiest, most unreliable piece of software we are running, and it's falling over (losing cluster state, hanging, crashing, etc) on an almost daily basis. We used to run 8 nodes, and it was a nightmare recovering from partitions. Now we've reduced it to 2 per cluster, and it's still a nightmare.