7 ms·
SurgeMQ: MQTT Message Queue at 750,000 MPS
- jacques_chester 12y agoThere's some typoes and broken links. I like the code, though, lots of work has been done. It's worth bearing in mind that messages per second is important, but it's easy to get fixated on benchmark porn. Different queueing systems have different guarantees, disciplines and semantics. These affect user-facing behaviour, which tends to be more important at first blush than throughput. I like zillions of messages per second as much as the next fellow. But frequently you need to worry about things like: * Are messages delivered once and only once? * Are messages are messages delivered at least once? * Can messages be dropped entirely under pressure? (aka best effort) * If they drop under pressure, how is pressure measured? Is there a back-pressure mechanism? * Can consumers and producers look into the queue, or is it totally opaque? * Is queueing unordered, FIFO or prioritised? * Is there a broker or no broker? * Does the broker own independent, named queues (topics, routes etc) or do producers and consumers need to coordinate their connections? * Is queueing durable or ephemeral? * Is durability achieved by writing every message to disk first, or by replicating messages across servers? * Is queueing partially/totally consistent across a group of servers or divided up for maximal throughput? * Is message posting transactional? * Is message receiving transactional? * Do consumers block on receive or can they check for new messages? * Do producers block on send or can they check for queue fullness? And there's probably a bunch more I've forgotten. The thing is that answers to these questions will fundamentally change both the functional and non-functional nature of your queueing system. For example, a queue system giving best-effort, unordered, non-durable behaviour is going to run a lot faster. It also pushes a lot of work onto the application programmer. On the other hand, once-and-only-once, durable, consistent queues are lot slower and screech to a halt under most partition conditions. But they also fit what most application developers expect to happen upon the first encounter with queueing systems. I work on a section of Cloud Foundry in my day job, and other teams have seen that different tasks require different queueing approaches. For example, stuff like metrics is is still useful under conditions of dropping messages, out-of-order messages and so on, because what's interesting is the statistics, not any one single measurement. But a message like "start this app" requires much higher guarantees of ordering, durability, delivery certainty. People get mad if your PaaS doesn't actually run the application you asked it to run. So, just remember: queues are not queues. You need to compare delivered apples with lossy oranges. As a note, the author observes that MQTT provides an option to select which delivery semantics you prefer (at-least-once, at-most-once / best-effort, once-and-only-once), but I can't see which one the benchmark is run for.
- mc_hammer 12y agoi was looking at rabbitmq - do you recommend a MQ that isnt a SaaS?
- ausjke 12y agoRabbitmq is AMQP and mostly an "enterprise-class" thing. MQTT is more of the IoT messaging protocol, don't think they're trying to do the same thing?
- mc_hammer 12y agoyea was just looking for an opinion - these guys know their stuff, i never get to leave the frontend so idk which mq to choose ;)
- jacques_chester 12y agoRabbitMQ is originally an installable package that can support a wide range of behaviours -- you still need to be mindful of which you choose. Depending on your settings, you can see orders of magnitude changes in throughput. Disclaimer: RabbitMQ belongs to Pivotal, the same company I work for.
- zhenjl 12y agoAuthor here. There's really no comparison tbh. RabbitMQ has been battle tested for years and SurgeMQ is just at its infancy. And RabbitMQ has a lot more enterprise security features that SurgeMQ doesn't have today. So if you are looking for a solution today, go w/ RabbitMQ. SurgeMQ hopefully will get there someday, but it's not ready today, yet.
- eternalban 12y ago> And there's probably a bunch more I've forgotten + Latency vs throughput. What would be really informative (and this is not remotely directed only at Jian Zhen's work here) is to see metrics across a spectrum per a given modality per a standard topology and capacity. This way it would be almost clear at a glance at which approach is the best fit for a given domain. Also I wonder if the time has come for the software geeks to visit their fellow geeks in the Mech-E departments and check out the work to date on flow analysis … ;)
- curiously 12y agoHow does this compare to RabbitMQ?
- zhenjl 12y agoAuthor here. There's really no comparison tbh. RabbitMQ has been battle tested for years and SurgeMQ is just at its infancy. And RabbitMQ has a lot more enterprise security features that SurgeMQ doesn't have today. So if you are looking for a solution today, go w/ RabbitMQ. SurgeMQ hopefully will get there someday, but it's not ready today, yet. Having said that, you can take a look at Tyler Treat's Dissecting Message Queues blog post. He did a fairly detailed performance comparison of the various MQs.
- itchyouch 12y agoThese numbers are great, but they are also highly dependant on hardware architecture and a strong coupling with network architecture. Another big contributor to messaging is in the message size. One can claim 10M messages/sec but for 1 byte messages, 10m messages/s is hardly impressive. I can say that on today's hardware, achieving 3-7M msgs/sec in a single-stream, guaranteed, in-order messaging is possible. This us also with 120 byte messages, which amounts to about 3-7Gbps if sustained messaging in clusters of 2 to 200+ machines. What do you need to achieve this? * Kernel bypass network architecture e.g. Solarflare 10/40gbe * 10gbe switches * decoupling of transport layer into a dedicated processor cores * busy-polling processes * business logic placed on dedicated cores. * use of shared memory and/or memory mapping to transport messaging between processes. It all sounds fairly convoluted to throw up processes and burn a core on rx/tx, but if you need to process millions of messages a second at fairly low latency, this is about the limit of where it goes. Using traditional application and socket tied to a single process will get you to around 1 million messages/sec with some very large buffers and efficient business logic, but thst eill be about it. The next step would be to start sharding streams, but that comes with its own complexities.
- zhenjl 12y agoAuthor here. The message size tested is 1024 bytes.
- eternalban 12y agoAre you cpu bound or (net) io bound?
- zhenjl 12y agoWhen I ran this I gave the server and client 2 cores each. Neither were maxing out the cores. At about 856K MPS (sent + received) @ 1024 bytes/msg, that's less than 1 GBps. So I can't imagine it being io(net) bound either. So inconclusive...unfortunately. Need to profile a bit more to know. Edit: Tho this is TCP based so it's possible that it's netio bound given the protocol overhead. At this message rate, each packet is almost guaranteed to be full size. So we are doing about 600K pps, approximately.
- jonquark 12y agoIt's good to see new MQTT servers. As one of the developers of IBM's product in this space I'd like to mention it: http://www-03.ibm.com/software/products/en/messagesight http://www-03.ibm.com/software/products/en/messagesight I'm biased and MessageSight has existed a lot longer so it's not fair to compare them but if people want to see numbers our performance report is here: http://www-01.ibm.com/support/docview.wss?uid=swg24035590 http://www-01.ibm.com/support/docview.wss?uid=swg24035590 (that report is for 1.1, the equivalent for 1.2 released a couple of weeks ago isn't out yet). Good luck and a fair wind to everyone in the MQTT space!