8 ms·

ZeroMQ: Modern & Fast Networking Stack

128 points by igrigorik 16y ago

eps 16y ago
Yup. I'm sure every network programmer who's worth his salt has a version of stackable abstraction layers library that runs of epoll or a variation thereof, and supports various transports from plain IP through domain sockets and to exotics like TLS over ICMP. It's like a rite of passage into the circle of network programming enlightenment :)
- stcredzero 16y ago
  I don't understand why HLL don't have some sort of standard abstraction on top of sockets. Is it because it's a fundamentally "leaky abstraction" or should it be done? Seems to me that most HLL attempts to abstract networking try to be too high level.
  - mhd 16y ago
    Plan 9 has dial[1] and friends, which kinda does to network connections what printf did for output. And, not surprisingly, something similar is in Go[2]. Of course, we're talking libraries here, so if you really want it in the language, then you're out of luck. Although any sufficiently advanced language should make this transparent enough anyway. [1]: http://swtch.com/plan9port/man/man3/dial.html http://swtch.com/plan9port/man/man3/dial.html [2]: http://golang.org/pkg/net/#Conn.Dial http://golang.org/pkg/net/#Conn.Dial
  - azim 16y ago
    Java EE has something very similar to 0MQ in the form of JMS. I'm just throwing out something random here, but I imagine the reason no one else really does this in standard libraries is because it's quite complicated and can easily lead to fragile protocols. Take AMQP, for example, it took something like 5 years to agree on their 1.0 standard.
    - riffraff 16y ago
      and """ While iMatix was the original designer of AMQP and has invested hugely in that protocol, we believe it is fundamentally flawed, and unfixable. """
    - stcredzero 16y ago
      Java EE has something very similar to 0MQ in the form of JMS. I've worked with JMS, and no. Maybe my implementation was flawed. (Weblogic) IIRC, you needed 5 different objects of 5 different classes just to do anything.
    - phintjens 16y ago
      JMS does cover some of the same ground, all messaging does. But it's basically an API wrapper around two ancient technologies, a queue system and a topic system. I think it might have been IBM MQSeries (which became WebSphere MQ) and some other product but can't find the details. What JMS achieved was to somehow turn these two very different semantics into a single one, based around "destinations". That was very clever but also makes JMS weirdly complex to use because the queue (request-reply) pattern and the topic (publish-subscribe) pattern just don't work the same way. We designed AMQP originally by taking the JMS spec and reverse engineering that into a wire level protocol. Then around version 0.3 we threw out destinations and came up with a generic wiring model based on exchanges, bindings, queues. That was my summer holiday in 2005. AMQP is BTW still some way away from a 1.0 standard and mainly it's been five years trying to get reliability working in the center of the network. That always seemed destined to failure, as I explained in a presentation to the working group in 2006: http://www.zeromq.org/whitepapers:switch-or-broker http://www.zeromq.org/whitepapers:switch-or-broker. AMQP and JMS both focus on resources held in the center of the network. It's familiar to anyone using HTTP as their stack. Thin stupid clients talking to big smart servers. 0MQ turns the focus to smart clients working across a thin, stupid (but massive) network. Both approaches have their value. We (iMatix and the 0MQ community) tend to believe we can do a lot more, faster and more cheaply, using the distributed approach.
- pjscott 16y ago
  Then they'd better find a new rite of passage, because ZeroMQ is turning into a category killer.
  - eps 16y ago
    So you are saying that the existence of a good library should discourage other people from writing their own? You are so missing the point.
    - sustrik 16y ago
      Did existence of TCP discouraged people to write their own implementation of OSI layer 4? Not really. But those who do so are a small -- i mean really small -- minority.
ekidd 16y ago
I've played with ZeroMQ on some small projects, and I've been quite impressed by it. Basically, ZeroMQ was designed to send millions of small, mostly-asynchronous messages at extremely high speeds. If you feed it messages very rapidly, it sends the first message immediately, and queues the rest in userspace until (IIRC) the network card is ready to transmit another packet. Then it sends all the queued messages as a batch. So you get both high throughput and low latency. There's some built-in support for common topologies (client/server, publish/subscribe, pipeline), and if you're willing to use the undocumented XREQ/XREP socket types, you can build more exotic topologies. Most of the value in ZeroMQ is the actual C++ implementation, so other languages generally just wrap the C++ code. The zeromq API is tiny. I haven't used it on a big production project yet, but I hope to do that soon.
- andrewvc 16y ago
  xrep/xreq really isn't well documented. I'm writing a learn by example series for ZeroMQ in Ruby, that already includes an XREP example. I'm planning on adding in XREQ this weekend. http://github.com/andrewvc/learn-ruby-zeromq http://github.com/andrewvc/learn-ruby-zeromq
  - shotgun 16y ago
    Thanks, man. Very helpful indeed!
- sshumaker 16y ago
  Well, the idea that simply 'queueing the rest in userspace' is OK illustrates a pretty disconcerting lack of understanding in writing networking code. This is not an acceptable solution in the real world. I want to correct that assumption right now - it's one of those places that higher level networking libraries often fall down - because simply 'fire and forget' is a leaky abstraction. Let's say you're streaming a large file over TCP to a client. Generally, disk reads are far faster than you can send data over the internet. If you naively keep sending data, you'll quickly fill up the OS buffers - and begin to consume memory as it's queued on the client side. In an extreme case, if you're sending a multi-GB file, you'll probably end up with most of the file sitting in RAM (assuming you don't run of ram and start thrashing). And for what? You really only need to load additional data in from the file as fast as you can send it in the socket. In a real-life case, streaming multiple files to multiple clients, you'll quickly run out of RAM with even moderate size files. Some high level libraries provide an event to let you know that the OS buffers have been emptied out. For example, Node.js has a drain event. But this has to be handled explicitly, and isn't really highlighted in the docs. But if you write your code naively, you will definitely run into problems. While it's great that many of these high level networking libraries have tried to abstract away many of the complexities of network programming, they're also a trap for the unwary. Just remember that they aren't a silver bullet that let you handle sending data over the internet as a simple function call.
mahmud 16y ago
ZeroMQ is quickly becoming an even bigger hammer in the premature optimization planet of Newbo-Thumbia. Edit: 1) It's a networking library; no admin tools or other soft handle-bars, like user-space utilities. 2) It uses a binary protocol. Good luck debugging that with syslog. It's a very powerful tool in the hands of a capable systems architect, who actually needs it. For the rest, it's pretty much like an adult male tiger; excellent to watch in its natural habitat from a safe distance, terrible pet idea for you and your fiancee (and not because you live in a studio apartment.)
- bretthoerner 16y ago
  Is it? I thought this was a level of abstraction higher? The premature optimizer would want access to the raw sockets this is built ontop of. I could be misunderstanding, though, I've never used it. The idea I have in my head is, ZeroMQ: Send a message, other side either receives full message as a single payload (or nothing at all? or an error is raised? I'm not tototally sure). Sockets: write(), read() and friends. Other side has to make sure he has the whole message, otherwise read() again and wait, deal with errors on its own, etc.
  - mahmud 16y ago
    I have seen higher-level libraries than Zero, I have seen middle-ware with better persistence than Zero, I have seen networking libraries better integrated with their languages than Zero. But I haven't seen a faster beast. It's right there in their <title/>: "Fastest. Messaging. Ever". The great majority of the people would just benefit from learning socket programming better, and sticking to something native to their tongue. Use the messaging infrastructure that integrates best with your development process, or opt for something with more padding, like Rabbit. Call me weak, but I like infrastructure software whose guts I can peek, lsof, cat, grep, log, strace and diff. Stuff that I can take a snapshot of and reproduce. Network services should be transparent to the application code base. I wanna poke a whole in my firewall, give some hint to inetd, a few lines of cron and be done with it. I simply don't trust my own software enough to be sure I will not screw up the infrastructure.
j_baker 16y ago
"Wouldn't it be nice if we could abstract some of the low-level details of different socket types, connection handling, framing, or even routing?" Perhaps I'm missing something, but wouldn't this make ZeroMQ a Modern & Fast abstraction of a Networking Stack?
- phintjens 16y ago
  If you read the 0MQ Reference Manual at http://api.zeromq.org/zmq.html http://api.zeromq.org/zmq.html you'll see that it specifies the abstraction, while the code implements that abstraction.
simplegeek 16y ago
Good article. Please consider changing the font-type and increasing the font-size.
- trevorcreech 16y ago
  Cmd+? We have the technology.
  - simplegeek 16y ago
    Thanks. I really didn't know that.
aaronblohowiak 16y ago
This is yet another well-written post.. i think Ilya has that rare talent for accurate and engaging technical writing. I'm consistently pleased with his work. Thanks.
mleonhard 16y ago
It looks like there's still no way for applications to detect when ZeroMQ encounters common networking problems. If my application can't differentiate between "no response received" and "network error", I'll end up re-implementing timeouts and error detection logic in the application protocol. In most situations that's a waste of time and adds extra weight to the system. No thanks.
- sustrik 16y ago
  That's the nature of networking. You can never differentiate between "network error" and "no response received". TCP in no better. You'll have accept that or keep with a single box.
- phintjens 16y ago
  In most cases 0MQ will recover silently (and usefully) from common networking problems. When a peer crashes, for example, and then comes back, its partners don't see the problem. Messages get queued, and then delivered. This works for the main socket types and transports (but not PAIR and inproc:) This lets us do things like start clients and THEN start a server... the clients automatically connect when the server comes along. The server can go away, be replaced by another on the same endpoint, and clients will gracefully and invisibly start talking to the new server. In some cases this is precisely what we want, in other cases it's not. If we need to detect dead peers, we add heartbeating as a message flow on top of 0MQ. Most larger 0MQ applications currently do this. Eventually 0MQ sockets may offer heartbeating, it seems a natural evolution. ("Seems" but is not necessarily.) Additionally there are some patterns (like synchronous request-reply) that simply don't handle network issues properly. If your service dies while processing a request, your client if it does a blocking read will hang forever. There are work-arounds such as using non-blocking reads. It would be a mistake to try to solve all challenges at once in any project. 0MQ is taking it step by step, starting with the core questions of how to design massively-scalable patterns correctly. Just doing that is worth gold, as you can see from people actually using 0MQ, who consistently tell us, "this makes our life orders of magnitude easier, thank you". And as we solve the core problems properly, we'll continue to address other aspects, either in 0MQ core or in layers on top of that, and this process will continue ad infinitum.

128 points by igrigorik 16y ago

eps 16y ago

Yup. I'm sure every network programmer who's worth his salt has a version of stackable abstraction layers library that runs of epoll or a variation thereof, and supports various transports from plain IP through domain sockets and to exotics like TLS over ICMP. It's like a rite of passage into the circle of network programming enlightenment :)

stcredzero 16y ago
I don't understand why HLL don't have some sort of standard abstraction on top of sockets. Is it because it's a fundamentally "leaky abstraction" or should it be done? Seems to me that most HLL attempts to abstract networking try to be too high level.
- mhd 16y ago
  Plan 9 has dial[1] and friends, which kinda does to network connections what printf did for output. And, not surprisingly, something similar is in Go[2]. Of course, we're talking libraries here, so if you really want it in the language, then you're out of luck. Although any sufficiently advanced language should make this transparent enough anyway. [1]: http://swtch.com/plan9port/man/man3/dial.html http://swtch.com/plan9port/man/man3/dial.html [2]: http://golang.org/pkg/net/#Conn.Dial http://golang.org/pkg/net/#Conn.Dial
- azim 16y ago
  Java EE has something very similar to 0MQ in the form of JMS. I'm just throwing out something random here, but I imagine the reason no one else really does this in standard libraries is because it's quite complicated and can easily lead to fragile protocols. Take AMQP, for example, it took something like 5 years to agree on their 1.0 standard.
  - riffraff 16y ago
    and """ While iMatix was the original designer of AMQP and has invested hugely in that protocol, we believe it is fundamentally flawed, and unfixable. """
  - stcredzero 16y ago
    Java EE has something very similar to 0MQ in the form of JMS. I've worked with JMS, and no. Maybe my implementation was flawed. (Weblogic) IIRC, you needed 5 different objects of 5 different classes just to do anything.
  - phintjens 16y ago
    JMS does cover some of the same ground, all messaging does. But it's basically an API wrapper around two ancient technologies, a queue system and a topic system. I think it might have been IBM MQSeries (which became WebSphere MQ) and some other product but can't find the details. What JMS achieved was to somehow turn these two very different semantics into a single one, based around "destinations". That was very clever but also makes JMS weirdly complex to use because the queue (request-reply) pattern and the topic (publish-subscribe) pattern just don't work the same way. We designed AMQP originally by taking the JMS spec and reverse engineering that into a wire level protocol. Then around version 0.3 we threw out destinations and came up with a generic wiring model based on exchanges, bindings, queues. That was my summer holiday in 2005. AMQP is BTW still some way away from a 1.0 standard and mainly it's been five years trying to get reliability working in the center of the network. That always seemed destined to failure, as I explained in a presentation to the working group in 2006: http://www.zeromq.org/whitepapers:switch-or-broker http://www.zeromq.org/whitepapers:switch-or-broker. AMQP and JMS both focus on resources held in the center of the network. It's familiar to anyone using HTTP as their stack. Thin stupid clients talking to big smart servers. 0MQ turns the focus to smart clients working across a thin, stupid (but massive) network. Both approaches have their value. We (iMatix and the 0MQ community) tend to believe we can do a lot more, faster and more cheaply, using the distributed approach.
pjscott 16y ago
Then they'd better find a new rite of passage, because ZeroMQ is turning into a category killer.
- eps 16y ago
  So you are saying that the existence of a good library should discourage other people from writing their own? You are so missing the point.
  - sustrik 16y ago
    Did existence of TCP discouraged people to write their own implementation of OSI layer 4? Not really. But those who do so are a small -- i mean really small -- minority.

ekidd 16y ago

I've played with ZeroMQ on some small projects, and I've been quite impressed by it. Basically, ZeroMQ was designed to send millions of small, mostly-asynchronous messages at extremely high speeds. If you feed it messages very rapidly, it sends the first message immediately, and queues the rest in userspace until (IIRC) the network card is ready to transmit another packet. Then it sends all the queued messages as a batch. So you get both high throughput and low latency. There's some built-in support for common topologies (client/server, publish/subscribe, pipeline), and if you're willing to use the undocumented XREQ/XREP socket types, you can build more exotic topologies. Most of the value in ZeroMQ is the actual C++ implementation, so other languages generally just wrap the C++ code. The zeromq API is tiny. I haven't used it on a big production project yet, but I hope to do that soon.

andrewvc 16y ago
xrep/xreq really isn't well documented. I'm writing a learn by example series for ZeroMQ in Ruby, that already includes an XREP example. I'm planning on adding in XREQ this weekend. http://github.com/andrewvc/learn-ruby-zeromq http://github.com/andrewvc/learn-ruby-zeromq
- shotgun 16y ago
  Thanks, man. Very helpful indeed!
sshumaker 16y ago
Well, the idea that simply 'queueing the rest in userspace' is OK illustrates a pretty disconcerting lack of understanding in writing networking code. This is not an acceptable solution in the real world. I want to correct that assumption right now - it's one of those places that higher level networking libraries often fall down - because simply 'fire and forget' is a leaky abstraction. Let's say you're streaming a large file over TCP to a client. Generally, disk reads are far faster than you can send data over the internet. If you naively keep sending data, you'll quickly fill up the OS buffers - and begin to consume memory as it's queued on the client side. In an extreme case, if you're sending a multi-GB file, you'll probably end up with most of the file sitting in RAM (assuming you don't run of ram and start thrashing). And for what? You really only need to load additional data in from the file as fast as you can send it in the socket. In a real-life case, streaming multiple files to multiple clients, you'll quickly run out of RAM with even moderate size files. Some high level libraries provide an event to let you know that the OS buffers have been emptied out. For example, Node.js has a drain event. But this has to be handled explicitly, and isn't really highlighted in the docs. But if you write your code naively, you will definitely run into problems. While it's great that many of these high level networking libraries have tried to abstract away many of the complexities of network programming, they're also a trap for the unwary. Just remember that they aren't a silver bullet that let you handle sending data over the internet as a simple function call.

mahmud 16y ago

ZeroMQ is quickly becoming an even bigger hammer in the premature optimization planet of Newbo-Thumbia. Edit: 1) It's a networking library; no admin tools or other soft handle-bars, like user-space utilities. 2) It uses a binary protocol. Good luck debugging that with syslog. It's a very powerful tool in the hands of a capable systems architect, who actually needs it. For the rest, it's pretty much like an adult male tiger; excellent to watch in its natural habitat from a safe distance, terrible pet idea for you and your fiancee (and not because you live in a studio apartment.)

bretthoerner 16y ago
Is it? I thought this was a level of abstraction higher? The premature optimizer would want access to the raw sockets this is built ontop of. I could be misunderstanding, though, I've never used it. The idea I have in my head is, ZeroMQ: Send a message, other side either receives full message as a single payload (or nothing at all? or an error is raised? I'm not tototally sure). Sockets: write(), read() and friends. Other side has to make sure he has the whole message, otherwise read() again and wait, deal with errors on its own, etc.
- mahmud 16y ago
  I have seen higher-level libraries than Zero, I have seen middle-ware with better persistence than Zero, I have seen networking libraries better integrated with their languages than Zero. But I haven't seen a faster beast. It's right there in their <title/>: "Fastest. Messaging. Ever". The great majority of the people would just benefit from learning socket programming better, and sticking to something native to their tongue. Use the messaging infrastructure that integrates best with your development process, or opt for something with more padding, like Rabbit. Call me weak, but I like infrastructure software whose guts I can peek, lsof, cat, grep, log, strace and diff. Stuff that I can take a snapshot of and reproduce. Network services should be transparent to the application code base. I wanna poke a whole in my firewall, give some hint to inetd, a few lines of cron and be done with it. I simply don't trust my own software enough to be sure I will not screw up the infrastructure.

j_baker 16y ago

"Wouldn't it be nice if we could abstract some of the low-level details of different socket types, connection handling, framing, or even routing?" Perhaps I'm missing something, but wouldn't this make ZeroMQ a Modern & Fast abstraction of a Networking Stack?

phintjens 16y ago
If you read the 0MQ Reference Manual at http://api.zeromq.org/zmq.html http://api.zeromq.org/zmq.html you'll see that it specifies the abstraction, while the code implements that abstraction.

simplegeek 16y ago

Good article. Please consider changing the font-type and increasing the font-size.

trevorcreech 16y ago
Cmd+? We have the technology.
- simplegeek 16y ago
  Thanks. I really didn't know that.

aaronblohowiak 16y ago

This is yet another well-written post.. i think Ilya has that rare talent for accurate and engaging technical writing. I'm consistently pleased with his work. Thanks.

mleonhard 16y ago

It looks like there's still no way for applications to detect when ZeroMQ encounters common networking problems. If my application can't differentiate between "no response received" and "network error", I'll end up re-implementing timeouts and error detection logic in the application protocol. In most situations that's a waste of time and adds extra weight to the system. No thanks.

sustrik 16y ago
That's the nature of networking. You can never differentiate between "network error" and "no response received". TCP in no better. You'll have accept that or keep with a single box.
phintjens 16y ago
In most cases 0MQ will recover silently (and usefully) from common networking problems. When a peer crashes, for example, and then comes back, its partners don't see the problem. Messages get queued, and then delivered. This works for the main socket types and transports (but not PAIR and inproc:) This lets us do things like start clients and THEN start a server... the clients automatically connect when the server comes along. The server can go away, be replaced by another on the same endpoint, and clients will gracefully and invisibly start talking to the new server. In some cases this is precisely what we want, in other cases it's not. If we need to detect dead peers, we add heartbeating as a message flow on top of 0MQ. Most larger 0MQ applications currently do this. Eventually 0MQ sockets may offer heartbeating, it seems a natural evolution. ("Seems" but is not necessarily.) Additionally there are some patterns (like synchronous request-reply) that simply don't handle network issues properly. If your service dies while processing a request, your client if it does a blocking read will hang forever. There are work-arounds such as using non-blocking reads. It would be a mistake to try to solve all challenges at once in any project. 0MQ is taking it step by step, starting with the core questions of how to design massively-scalable patterns correctly. Just doing that is worth gold, as you can see from people actually using 0MQ, who consistently tell us, "this makes our life orders of magnitude easier, thank you". And as we solve the core problems properly, we'll continue to address other aspects, either in 0MQ core or in layers on top of that, and this process will continue ad infinitum.