28 ms·

BlazingMQ: High-performance open source message queuing system

695 points by carride 3y ago

eightysixfour 3y ago
What a phenomenal landing page. Thanks to the animations I have a good idea of what this does and how it does it. You barely need to know what message queuing is to understand it.
- ElongatedMusket 3y ago
  Getting started pages are great too, with guided examples and use cases: https://bloomberg.github.io/blazingmq/getting_started https://bloomberg.github.io/blazingmq/getting_started BMQ license is apache 2.0 for anyone curious
- scottlamb 3y ago
  The animations are very slick, but I can still grumpily pick at them: the "Clustering and Quorum-based Replication" one shows 4 total servers (suboptimal number because quorum requires 3/4 where I'd prefer to require 2/3 or 3/5) and all the followers acking before the primary proceeds (the animation could show one lagging instead).
  - amelius 3y ago
    Also, why is the middle replica special, in that the message is passed through it in the end?
  - BeeOnRope 3y ago
    4 might be useless compared to 3 from a "required quorum" PoV but if you want to stay up with 1 failure and the load can't be handled on 3 servers but 4 is OK, it is optimal isn't it? I.e. selection of ideal server count is a multi-dimensional problem.
    - scottlamb 3y ago
      Worse than useless from a quorum perspective, because you've increased the number of servers that have to participate in quorum (quickly) without increasing the number that don't. Looking at it the other way, you go from having problems when 2/3 are slow/broken to having problems when 2/4 are slow/broken. (also note slow doesn't even mean the machine is having problems; in geographically distributed setups it may just be due to physical distance from the leader.) Yes, 4 servers might handle more load than 3 servers [1], but so would 5 servers. I'd guess that if you've got enough message queue traffic that 3 servers can't handle the load, those servers aren't a significant fraction of your total cost. You might as well go to 5. Likewise, if 5 aren't enough, I'd skip over 6 and go to 7. If 7 aren't enough...I'd probably rethink my design entirely...but I'd prefer 9 to 8. Alternatively, quorum systems can support "zero-weight replicas" which effectively don't vote but still can be useful for eventually consistent reads. Then you have this server that helps handle the load but doesn't increase the number of servers that have to participate in a round. [1] if the load is overwhelmingly eventually consistent read operations and their multi-hop topology with proxies doesn't sufficiently reduce the load on the replicas. It's not obvious to me if this scenario is plausible or not.
- gr33nq 3y ago
  Does anyone recommend any particular piece of software for creating these types of visualizations with relative ease? I've always been a fan of animations for helping to understand some technical topics.
  - erwinkle 3y ago
    They use anime.js for their animations https://animejs.com/ https://animejs.com/ As seen in their source code here: view-source:https://bloomberg.github.io/blazingmq/assets/animations/animations/bmq/message-paths/index.html https://bloomberg.github.io/blazingmq/assets/animations/anim...
  - maxwellito 3y ago
    As erwinkle mentioned, these animations use anime.js If you want to build something but don't really know how to do it, I would suggest the following: 1. Build the visual content in an SVG with Inkscape. It should be easy for you to build the layout with assets/item to animate. Don't forget to group items and set names so you can reference them later. 2. Once exported, insert the SVG in an HTML page and add anime.js. This library will let you animate the content you just created (you should be able to reference your named items from Inkscape and animate them). The learning curve might be tough depending on your experience regarding animation (or CSS in general). Good luck :)
  - treprinum 3y ago
    animatron.com for animated SVG or JS. I used it for some algorithm animations myself.
  - danielvaughn 3y ago
    I've heard Framer can make some impressive animations, but I don't know much about how well they export to web. What's on display here is really cool, it's all animated SVG.
  - jasonlotito 3y ago
    Okay, this is going to sound weird, but I've used Keynote in the past. I'm sure PowerPoint can do this as well, but Keynote has Magic Move, and you can export this animation. It's very easy to work with, and do simple animations. I'm sure there are much better tools for this, but as someone who knew Keynote, this was always effective for me.
- rewmie 3y ago
  I agree, this is perhaps the very best landing page I ever saw. What a treat.
- nine_zeros 3y ago
  Agreed. This feels like a page created from enthusiasm, not from a place of promo-seeking performance review.
hkt 3y ago
It has a gorgeous landing page but I'm not really sure why this is better than any other MQ. Can someone more knowledgeable provide any insight into its comparative advantages? Edit: I did see the comparisons page, but.. well, there's more to life than Rabbit and Kafka!
- traceroute66 3y ago
  > Can someone more knowledgeable provide any insight into its comparative advantages? Well, you know, there is the "Comparison With Alternative Systems" page[1]. :-) Sadly it doesn't include NATS in the comparison. I would have been interested to see that. But the usual suspects (Rabbit & Kafka) are given a mention. [1] https://bloomberg.github.io/blazingmq/docs/introduction/comparison/ https://bloomberg.github.io/blazingmq/docs/introduction/comp...
  - rlonstein 3y ago
    I'd also be interested in seeing how it compares to AMQP implementations (Apache Qpid for example) and any of the proprietary message queue systems (IBM and Microsoft) they refer to but don't name-check.
- vyskocilm 3y ago
  FJI: There is a faq entry about that https://bloomberg.github.io/blazingmq/docs/introduction/comparison/ https://bloomberg.github.io/blazingmq/docs/introduction/comp...
- Cthulhu_ 3y ago
  I mean one unsubstantiated armchair take: it's newer than Rabbit / Kafka (RabbitMQ was built in 2007, Kafka in 2011), since then the learnings from distributed systems, message queues and programming languages has improved by leaps and bounds, and the existing ones will have backwards compatibility / legacy issues. That said, there are indeed plenty of commercial and noncommercial alternatives, so now that it's been published I'm sure there will be some thorough people making comparisons soon!
- MathMonkeyMan 3y ago
mgaunard 3y ago
I don't understand how you can make queueing high-performance. Queueing things is literally the opposite of making things fast.
- LoganDark 3y ago
  Queueing things can result in better throughput because you don't have to wait for the other side to process your last message before you can begin work on the next one.
  - jeffbee 3y ago
    That's just buffering. A lot of people use Kafka this way, as an impedance adapter for a producer with an internal architecture that can't tolerate blocking on production. Of course this requires you to unrealistically assume that Kafka is a zero-impedance message sink. But what I think the other post is alluding to is the fact that in-order delivery is often not a requirement, and can be an anti-feature. I know that in every use of Kafka I have personally encountered the relative order of messages was quite irrelevant but the in-order implementation of Kafka converts the inability to process a single record into an emergency.
    - therealdrag0 3y ago
      There’s a lot of brokers without ordering guarantees. ActiveMQ for example.
- hbarka 3y ago
  1,000,000 messages per second isn’t high performance?
  - mgaunard 3y ago
    A message is typically 1k. That would mean 1GB/s. That is decent but still below the speed of the network, which should be the main blocker.
- 3y ago
fidotron 3y ago
That fan-out functionality is particularly interesting for anyone needing to implement the upstream part of a firehose API, which of course Bloomberg do.
arnold_palmur 3y ago
I wonder how this compares to something like Aeron performance-wise - though Aeron is strictly messaging (not queueing).
fizwhiz 3y ago
> This section explains the leader election algorithm at a high level. It is by no means exhaustive and deliberately avoids any formal specification or proof. Readers looking for an exhaustive explanation should refer to the Raft paper, which acts as a strong inspiration for BlazingMQ’s leader election algorithm. So their own homegrown leader election algorithm? > BlazingMQ’s leader election and state machine replication differs from that of Raft in one way: in Rafts leader election, only the node having the most up-to-date log can become the leader. If a follower receives an election proposal from a node with stale view of the log, it will not support it. This ensures that the elected leader has up-to-date messages in the replicated stream, and simply needs to sync up any followers which are not up to date. A good thing about this choice is that messages always flow from leader to follower nodes. > BlazingMQ’s elector implementation relaxes this requirement. Any node in the cluster can become a leader, irrespective of its position in the log. This adds additional complexity in that a new leader needs to synchronize its state with the followers and that a follower node may need to send messages to the new leader if the latter is not up to date. However, this deviation from Raft and the custom synchronization protocol comes in handy because it allows BlazingMQ to avoid flushing (fsync) every message to disk. Readers familiar with Apache Kafka internals will see similarities between the two systems here. "a new leader needs to synchronize its state with the followers and that a follower node may need to send messages to the new leader if the latter is not up to date". I thought a hallmark of HA systems was fast failover? If I come to your house and knock on the door, but it takes you 10mins to get off the couch to open the door, it's perfectly acceptable for me to claim you were "unavailable". Pedants will argue the opposite.
- anentropic 3y ago
  "deliberately avoids any formal specification or proof" how is this possibly a selling point?
  - scottlamb 3y ago
    > "deliberately avoids any formal specification or proof" > how is this possibly a selling point? Context. In an overview doc? The informal version is accessible to more audiences. As the canonical design doc for the system? It's certainly not.
i-use-nixos-btw 3y ago
Off topic here but > Carefully architected and written in C++ from the ground up with no dependency on any external framework, BlazingMQ provides… What a weird selling point. There are pros and cons to having no dependencies. Not long ago, it was a common decision for C++ projects because dependency management was a mess. But that hasn’t been the case for a long time (arguably we have the opposite problem - there are so many ways of doing it: Conan, vcpkg, bazel, spack, raw cmake, nix, etc) So what would the pros and cons be today and why is it a selling point? For instance, a pro is that everything is bespoke to the task at hand, no hammering square pegs into round holes designed for a slightly different use case. A con is that the entire attack surface is managed by one team. CVEs identified and solved on another project is a pretty good thing when you depend on that project. The main reason I’m surprised, though, is that there are some no-brainer dependencies these days. Fmt, catch2/gtest, metaprogramming libraries, etc.
- omeid2 3y ago
  Sending patches to fix a project is hard work, sending patches to fix issues in a projects' upstream projects is even harder. This alone is an important point to avoid dependencies, specially if you're anything like Bloomberg.
  - i-use-nixos-btw 3y ago
    I run into that problem a lot, to be fair. Though my approach is to fork and maintain the fork until my PR is accepted, merged, and in a tagged release. Another is to apply a diff to the dependency within the build system, which is what I do a lot in Nix (mainly to solve cmake issues on hermetic builds). Either way, it isn't unsurmountable. It doesn't really matter so much if it's used as an executable rather than as a library, though in the case of the latter it's really handy to be able to e.g. pass a custom spdlog sink for logging.
- deleted 3y ago
  [deleted]
- callbacker 3y ago
  Hi, one of the authors here. BlazingMQ depends on two other open source C++ libraries: https://github.com/bloomberg/bde https://github.com/bloomberg/bde and https://github.com/bloomberg/ntf-core https://github.com/bloomberg/ntf-core. I believe documentation writer wanted to highlight that BlazingMQ does not depend on frameworks like ZooKeeper, etc.
notjoemama 3y ago
It doesn't use UDP. So my tongue-in-cheek reply is, it doesn't qualify to have "blazing" in the name. Hopefully that's not just funny inside my own head. :) https://bloomberg.github.io/blazingmq/docs/architecture/network_transport/ https://bloomberg.github.io/blazingmq/docs/architecture/netw...
- rewmie 3y ago
  > It doesn't use UDP. So my tongue-in-cheek reply is, it doesn't qualify to have "blazing" in the name. Arguably, if you start with UDP but you also need to ensure retransmission and reordering, you eventually end up reimplementing TCP but poorly.
  - jayd16 3y ago
    I guess you could implement QUIC instead.
  - lordnacho 3y ago
    Check out Aeron, they have a way to multicast (thus UDP) reliably and ordered. Based on negative acks. Super fast.
    - HappySweeney 3y ago
      NORM from the Naval Research Labs is another implementation of the same concept. https://github.com/USNavalResearchLaboratory/norm https://github.com/USNavalResearchLaboratory/norm
    - jonathanoliver 3y ago
      Aeron is another awesome project from Martin Thompson (Disruptor, Simple Binary Encoding, Mechanical Sympathy, etc.). That guy really knows his stuff regarding performance.
    - phishhook 3y ago
      Sounds like PGM
say_it_as_it_is 3y ago
Why is Bloomberg approving the open sourcing of this system?
- esafak 3y ago
  They must not consider it a trade secret, so they can earn good will, generate public contributions, and advertise their brand by sharing it.
- SkipperCat 3y ago
  If it gains wide adoption, then they've got a global community of people providing free software engineering for their messaging platform. Its a win-win.
- loire280 3y ago
  Why wouldn’t they? This technology is not a competitive advantage for their company, wouldn’t make sense for them to sell as a standalone product, and it helps their engineering department attract good talent.
- BossingAround 3y ago
  Hiring strategy--there are a ton of people who'll take a slightly lower salary if they can get a public record of their code commits.
- Cthulhu_ 3y ago
  Because they're not making money off of their queueing system, but what they build with it; to them it's a cog in their machine, and open sourcing it means others may adopt it and help them maintain and improve it.
- puchatek 3y ago
  Same reasons any company open sources their software
- sdesol 3y ago
  Here's some community stats for Bloomberg open source projects https://devboard.gitsense.com/bloomberg https://devboard.gitsense.com/bloomberg For the 200 repos that they have made open source, they had over 6000 people contribute feedback and/or code. So in addition to attracting talent, they also get more testers and feedback. Full Disclosure: This is my tool
insanitybit 3y ago
> Supports a unique multi-hop network topology leading to a distribution tree for each queue, thereby leading to network bandwidth savings for certain use cases. This is very interesting. One of the benefits of Kafka is partition routing so I'm curious to learn how this may compare.
- Sparkyte 3y ago
  No idea, but I am interested too. One of the reasons this was probably created was because of a network constrained environment where data is expensive in more than one way. Cloud Providers can be expensive.
  - eric-hu 3y ago
    > Cloud Providers can be expensive. Do you have a story around this? If so, would love to hear more about it.
    - lelandbatey 3y ago
      Specifically, egress from Cloud Providers (transmitting data from inside the cloud to a computer outside the cloud) is the usual culprit for being "very expensive". It's part of a pricing strategy by cloud providers that encourages folks to put all their infrastructure inside of the cloud and to have none of it outside the cloud. As one example of the ink spilled over this topic, see this discussion from 2 years ago about AWS's very high egress fees: Article "AWS’s Egregious Egress" (2021) https://blog.cloudflare.com/aws-egregious-egress/ https://blog.cloudflare.com/aws-egregious-egress/ HN discussion: https://news.ycombinator.com/item?id=27930151 https://news.ycombinator.com/item?id=27930151
      - beyonddream 3y ago
        Ingress can also be costly especially if there is a steady state of high load traffic sent from on-prem machines to machines inside cloud (not sure about aws but have experienced this with azure where we had to resort to buy their expressroute which was very costly and ultimately unsustainable for us)
davidkunz 3y ago
It's called BlazingMQ but not written in Rust?!
- reactordev 3y ago
  I’ll admit, I went into their repo expecting to find Rust, or at the very least go. Pleasantly surprised to see C++ code though I would need to look further to see if it’s “good” C++ code. The naming conventions of the source tree lends me to suspect it’s not.
  - deleted 3y ago
    [deleted]
  - beanjuiceII 3y ago
    i too determine if code is good based off its naming conventions and other things like my linter is set to enforce MiXeD_SpOnGeBoB-KeBaB_CaSe as the default casing
    - gabereiser 3y ago
      Better than p_f_var_t;
- Cthulhu_ 3y ago
  The language is an implementation detail; what makes you think Rust would be better suited for such a specialist piece of software?
  - higherhalf 3y ago
    It's a joke, as some Rust projects used to repetitively claim to be "memory safe & blazing fast", thus becoming a tongue-in-cheek phrase. I also anticipated language to be Rust, but it would be way too comical. https://github.com/mTvare6/hello-world.rs https://github.com/mTvare6/hello-world.rs
29athrowaway 3y ago
420 messages per second.
- rurtrack 3y ago
  Sweet
  - 29athrowaway 3y ago
    Dude
    - deleted 3y ago
      [deleted]
endisneigh 3y ago
it never ceases to amaze me how the software engineering profession has very smart folks reimplement the same conceptual primitives over and over again. I'm curious if hospitals (if that's even analogous to a company) do similar things by having different processes to solve the same problem. I imagine this is used for Bloomberg (the terminal) and not Bloomberg (the website)? going back to the article - fantastic animations. I'm just as curious to how that was made as the queue itself.
- BossingAround 3y ago
  > I'm curious if hospitals (if that's even analogous to a company) do similar things by having different processes to solve the same problem. Of course they do, unless the processes are regulated by a country (such as handling highly addictive opiates in the US).
  - quercusa 3y ago
    In the US, all hospitals are required to use electronic health records (EHRs) and most are implemented in Epic, the leading EHR system. But they are almost always electronic implementations of their old paper record systems, dating back to who-knows-when.
  - thfuran 3y ago
    Yeah, pretty much any medium to large company is going to have a bunch of processes for doing things that any company that size needs to do and some more that are a bit more industry/domain specific but still basically the same as those at every other company in the space.
- Sparkyte 3y ago
  Part of this evolution is from funded development to opensource development. Then there is language variances like flavors so what ends up happening is a million different conceptual primitives that fundamentally the same. It isn't the same because of a few things. At the end of the day too it is also about how good the support is over the rest. If a new MQ can demonstrate better tooling nothing stops a company from adopting it.
mkl95 3y ago
How does it compare to NATS.io?
- leetharris 3y ago
  It is amazing to me that NATS.io is not more popular. It is so much better than Kafka, extremely simple, and extremely light.
  - slantedview 3y ago
    Nats is simple because it doesn't bother addressing some of the scalability problems that Kafka was designed to address, like clustering with partitions: https://docs.nats.io/legacy/stan/intro/partitioning https://docs.nats.io/legacy/stan/intro/partitioning
    - coder543 3y ago
      You linked to NATS Streaming, which has been deprecated for years. It’s not relevant. It can still be used for legacy code that still needs it, as the URL even shows, but it is not recommended for new applications. The current standard is NATS JetStream, which absolutely addresses scalability, arguably to an even larger degree than Kafka, since it can be used to create a globally interconnected supercluster of clusters where messages can be configured to flow between regions as needed. I doubt anyone would run a Kafka cluster that spans multiple regions.
      - kitd 3y ago
        I doubt anyone would run a Kafka cluster that spans multiple regions. Stretch clusters and clusters with brokers in different AZs are absolutely a thing in Kafka. I make no comment on the ease relative to Nats, which I don't know at all.
        coder543 3y ago
        I was talking about different regions altogether, not just different AZs. My understanding is that it’s generally a bad idea in Kafka due to the latency, but I could be wrong. EDIT: Definitely sounds like there are some tight latency requirements[0]: > A stretched 3-data center cluster architecture involves three data centers that are connected by a low latency (sub-100ms) and stable (very tight p99s) network, usually a “dark fiber” network that is owned or leased privately by the company. 100ms would theoretically be enough to span the contiguous United States, but the references to "very tight p99s" and "dark fiber" make me wonder if 100ms is actually acceptable, or just a theoretical maximum that can only be allowed under absolutely perfect conditions. Either way, suggesting the user should have access to dark fiber between their regions does not fill me with confidence about the robustness of this solution. I'm sure it would be fine for geographically close datacenters, as AZs are designed to be. [0]: https://docs.confluent.io/platform/current/multi-dc-deployments/multi-region-architectures.html#stretched-cluster-3-data-center https://docs.confluent.io/platform/current/multi-dc-deployme...
mizzlr_ 3y ago
LavinMQ is much better.
- konart 3y ago
  Better how?
- renewiltord 3y ago
  Appears 10^3 times latency for 1 KiB message according to here but better at lower 16 B size https://www.cloudamqp.com/blog/lavinmq-benchmarks.html https://www.cloudamqp.com/blog/lavinmq-benchmarks.html Seems situation dependent.
  - phishhook 3y ago
    Does that do HA/replication/clustering etc though?
Thaxll 3y ago
Which version of C++ is this built on?
- coder543 3y ago
  One person opined that it is similar to C++03: https://news.ycombinator.com/item?id=36898899 https://news.ycombinator.com/item?id=36898899
- hallfox 3y ago
  Hi, dev here. BlazingMQ is written with a baseline of C++03, though our docker tutorial builds it with C++17.
- frutiger 3y ago
  It targets AIX and Solaris (in addition to Linux) and vendor-supported C++ toolchains have spotty support for C++11 and newer.
user6723 3y ago
[flagged]
- Cthulhu_ 3y ago
  I don't understand the language preference; it's an implementation detail.
synergy20 3y ago
how does this compare against other existing message queuing systems? the github shows its codebase is in c++ with gcc-7.3, that means it might be using c++14 in the code, or even c++17, which is nice.
eska 3y ago
I checked the docs, but couldn’t find anything about mqtt compat?
- m00x 3y ago
  I don't think this is an MQTT broker.
renewiltord 3y ago
Very interesting. Thank you for benchmarks. Thank you for sharing. In a zero-replication no-leader-election (or all-in-one-node) scenario what is min latency observed? It appears predominant delay is in ACKing to ensure durability, right? C++ and Java client examples are very simple which is cool. Would appreciate if there is hook to enable `SO_TIMESTAMPNS` on socket and retrieve `cmsg` to get accurate timestamping and permit client code benchmarking.
- callbacker 3y ago
  Hi, one of the authors here. > In a zero-replication no-leader-election (or all-in-one-node) scenario what is min latency observed? We saw sub-millisecond latency in this case (typically double-digit microseconds).
  - renewiltord 3y ago
    Splendid! Great work, guys. And thanks again for sharing. Dockerfile is good way to share build deps.
    - callbacker 3y ago
      Thank you!
1letterunixname 3y ago
Marketing is all well and good. Let's see the data of p99 latency and throughput numbers compared to rabbitmq, kafka, activemq, qpid, and hornetq. Also, it's worth identifying and setting aside use-cases for broker-less messaging where 0mq, nanomsg, etc. could be more appropriate. Edit: There's only published performance data of itself, which has little value: https://bloomberg.github.io/blazingmq/docs/performance/benchmarks/ https://bloomberg.github.io/blazingmq/docs/performance/bench...
glonq 3y ago
Are they working on other API's besides Java and C++?
- frakkingcylons 3y ago
  The python client library is planned to be published. They say they need to remove some dependencies and update the workflow to something more suitable for OSS.
- callbacker 3y ago
  Hi, one of the authors here. We will be releasing a Python client library in a few weeks.
  - KrishnaAnaril 3y ago
    Any plans to support .NET?
    - mikece 3y ago
      There are a number of ways to wrap native code libraries as a managed library. I did it once (packaged an Obj-C library as a .NET DLL for a Xamarin app) back in 2018 but had lots of help from a teammate who knew that dance far better than me and I forgot all of the details. I googled and found at least four different blogs posts showing how to do it. I know that's not an answer but if you're feeling adventurous you might be able to help the project.
      - glonq 3y ago
        +1 vote from me for a first-class client for .Net core. I'd tolerate wrappers for certain things, but not for this thing.
    - callbacker 3y ago
      Hi, no plans as of yet. Please also see my response to the Golang client library request above.
alberth 3y ago
> “Carefully architected and written in C++” Amazing to see a non-Rust product posted to HN.
- geertj 3y ago
  I’ve been getting pretty deep in C++ recently and the name Bloomberg shows up all across the C++ standards community. They seem to be a strong C++ shop.
  - tkhattra 3y ago
    yep, the authors of "Embracing Modern C++ Safely" [1] are Bloomberg folks [1] https://vittorioromeo.info/emcpps.html https://vittorioromeo.info/emcpps.html
- qwertox 3y ago
  True, but maybe its age could play some role in it being a C++ project. > BlazingMQ is an actively developed project and has been battle-tested in production at Bloomberg for 8+ years. https://github.com/bloomberg/blazingmq https://github.com/bloomberg/blazingmq
- harerazer 3y ago
  Recently, mold 2.0 was posted, which is also C++.
- beanjuiceII 3y ago
  don't worry the army of Rust authoritarians are waiting to reply as we speak. how software is just not ethical or correct unless its in rust
tetrep 3y ago
Is this implemented with any sort of thought to security? I don't see anything that implies as such, all FAQ and comparisons to other message queues say nothing of security (I skimmed and ctrl+f so I could have missed it). This doesn't bode well for a network-based application written in a memory unsafe language. edit: I'm surprised so many people are interpeting this as a weird cargo cult thing. Application security includes a lot of mundane and critical things like: Authentication, authorization, message signing / authentication (e.g. HMAC), encryption, secret/key management, how the system handles updates, etc. You can read a bit more about it here: https://en.m.wikipedia.org/wiki/Application_security https://en.m.wikipedia.org/wiki/Application_security
- mrits 3y ago
  Would you prefer it written in Rust with everything wrapped in unsafe block?
  - tetrep 3y ago
    I care more about methodology than specific methods. I would like applications that handle I/O, especially network I/O, to be designed and built with security in mind. I asked my question because that doesn't appear to be the case here, and I think that is dissapointing and concerning from a security perspective.
  - coder543 3y ago
    That is an obviously false dichotomy. The author of the comment never mentioned Rust, and wasn’t asking about Rust. Such a false dichotomy feels like flamebait, which is against HN guidelines. Your false dichotomy also implies that “unsafe” blocks are as unsafe as C++, which is not true. “Unsafe” in Rust turns off very few checks[0], most of them are still active. No one would write a serious Rust program entirely in unsafe anyways. Regardless, asking about security considerations is a valid thing to do, even if it were written in Rust. Security is not just about memory safety. [0]: https://doc.rust-lang.org/std/keyword.unsafe.html#unsafe-abilities https://doc.rust-lang.org/std/keyword.unsafe.html#unsafe-abi...
fadedsignal 3y ago
A yet another MQ :-) I quickly checked the performance page and it looks much better than its competitors. Documentation is better as well.
- MathMonkeyMan 3y ago
  Last time I checked, they weren't even building with optimization, and it was still the fastest thing in the West. Peeved me, but whatever. "Every byte counts" was the motto of the team that designed it.
  - deleted 3y ago
    [deleted]
travisgriggs 3y ago
Interesting that there's a comparison (albeit low on details) to Rabbit, but not MQTT? Anyone had experience with both MQTT and BlazingMQ able to compare/contrast?
- dbrueck 3y ago
  Isn't MQTT more of a standard than an implementation?
  - winrid 3y ago
    Yes. They prob mean rabbitmq.
  - smarx007 3y ago
    I would be interested in the differences between BlazingMQ, AMQP, and MQTT protocols and the invariants afforded to the clients, not the implementations per se.
  - sheerun 3y ago
    I think he means implementation of MQTT protocol, like https://mosquitto.org/ https://mosquitto.org/
sitkack 3y ago
TLA 0.1% Finally! Thank you!
- callbacker 3y ago
  Thanks! The spec can be found here: https://github.com/bloomberg/blazingmq/tree/main/etc/tlaplus https://github.com/bloomberg/blazingmq/tree/main/etc/tlaplus
weekendcode 3y ago
Built in C++, awesome :)
- sho 3y ago
  Is that really your reaction to C++? These days when selecting a critical infrastructure component I'm very inclined to prefer memory-safe by default, like rust or (somewhat less so) golang. Sure, C/C++ projects that have been around and under attack for years (redis, postgres, etc) are fine but they've had those years of battle testing. For a new project I really feel a lot safer if they're built in something more failsafe.
  - wiseowise 3y ago
    Can you show me where modern C++ is unsafe and why it matters in this specific context?
    - epinephrinios 3y ago
      Read this - https://docs.google.com/document/d/e/2PACX-1vRZr-HJcYmf2Y76DhewaiJOhRNpjGHCxliAQTBhFxzv1QTae9o8mhBmDl32CRIuaWZLt5kVeH9e9jXv/pub https://docs.google.com/document/d/e/2PACX-1vRZr-HJcYmf2Y76D...
nodesocket 3y ago
Awesome landing page and looks great. Documentation is very well done. Would love to see more clients (python) and a web u/i.
- callbacker 3y ago
  Hi, one of the authors here. We will be releasing a Python client library in a few weeks.
melenaboija 3y ago
I have been working on an engine to perform heavy computations that is distributed and my messaging choice has been gRPC + Flatbuffers + proxys and I was wondering if someone could name some reasons or use cases to chose message brokers vs a lower level approach like gRPC. Is it mostly because of synchronous/asynchronous needs or there is something else such as latency or even ease of implementation?
- ozarker 3y ago
  Depends on your use case. Queueing messages enables the competing consumer flow which implicitly allows load-leveling which is great if your traffic is spiky and you need a little time to auto-scale or other scenarios like that. If you’re just looking to blast out data to all consumers or just load-balancing is fine, approaches like yours are probably better.
- imachine1980_ 3y ago
  grpc client a -> grpc server a for client a grpc client b -> grpc server a for client b grpc client a -> message broker(implemen a) --> grpc client b -> message broker(implemen b) --> message broker-> your app for the broker this make sense if dont make sense manage the quque in your app if ypu want to abstract the implementations is worth if you have n to 1 or n to n and all works different
- pravus 3y ago
  If your workers can hop onto the next task without having to reach out for any additional state or report back to anything synchronously a distributed log/spool/queue/broker works well. If you need light-weight reporting like a status code you can use RPC/reply mechanisms if the broker supports it (e.g. rabbitmq), build your own (custom headers), or use an API/gRPC call. Everything else starts needing some sort of more sophisticated coordination mechanism and gRPC will look attractive for that although there are usually other options as well.
- stolsvik 3y ago
  It is the asynchronous and decoupled nature of messaging that pulls architectures towards message brokers. In a "MOM Architecture", the producing of messages, and consuming of messages, is completely decoupled - both in "space" (HA, location transparency, replication etc) and in time (the async nature). This leads to a whole heap of benefits. I've outlined a few of them here, in the "Rationale for Mats3", my Message-Oriented Async RPC library: https://mats3.io/background/rationale-for-mats/ https://mats3.io/background/rationale-for-mats/ You will find that the Reactive Manifesto also lays out the same type of arguments (there's some links to it from the list in the first link): https://www.reactivemanifesto.org/ https://www.reactivemanifesto.org/ and its glossary: https://www.reactivemanifesto.org/glossary https://www.reactivemanifesto.org/glossary
MathMonkeyMan 3y ago
They actually open sourced it! I worked on this team very briefly. Great team working on some interesting tech.
- deleted 3y ago
  [deleted]
- theptip 3y ago
  Would love to hear any technical history you can share, what did this replace, and what shortcomings led to it being built? (Seems quite similar to Kafka but perhaps more emphasis on low latency rather than raw throughput?)
  - callbacker 3y ago
    Some of your questions might be answered in the accompanying blog - https://www.bloomberg.com/company/stories/bloomberg-publishes-blazingmq-modern-high-performance-message-queuing-system https://www.bloomberg.com/company/stories/bloomberg-publishe..., and the comparison doc https://bloomberg.github.io/blazingmq/docs/introduction/comparison/ https://bloomberg.github.io/blazingmq/docs/introduction/comp....
accrual 3y ago
Does anyone here do large scale interface stuff with many millions of messages? I do this in healthcare and would like to try applying my skills somewhere else...
- didip 3y ago
  Just about any large tech companies and unicorns have a large queue (or many of them).
- matthewpersico 3y ago
  https://careers.bloomberg.com/job/search https://careers.bloomberg.com/job/search :-)
Eumenes 3y ago
As much as Bloomberg (the guy and company) creep me out, this is impressive.
- midoridensha 3y ago
  I have a similar feeling about Facebook, but zstd is amazing.
kajaktum 3y ago
I don't doubt that it is fast but please don't name your thing <claim><thing>.
- seabrookmx 3y ago
  Not a big FastAPI fan eh? Or MongoDB? What about BigQuery? So many bad names!
- wiseowise 3y ago
  Why?
  - kajaktum 3y ago
    Because the name is a technical claim. Is it fast today? Maybe but will it still be considered "blazing" in 20 years? This is equivalent to naming your thing VeryFastMQ. I can already imagine people saying "Oh yea BlazingMQ is craaaawling right now, I am thinking of replacing it with SuperiorMQ".
favflam 3y ago
From the "leadership election" snippet, this setup looks perilously close to the peer to peer propagation of transactions and blocks in a blockchain system such as Solana. I suppose the latency requirements are much tighter than 400ms though. I know Solana does mutual TLS over QUIC authentication between peers and does bandwidth rate limiting via stake. Also, the topology is self organizing. Is it worth copying this tech into a message queue system?
k2xl 3y ago
Congrats on open sourcing this, but I am curious if anyone here has ever had an actual scale issue that required changing from some "slower" queuing system (like RabbitMQ, Kestrel, Kafka, etc) where you actually needed to change queuing systems? Maybe I'm old school but 99% of the use cases I've worked with in my 20 year career could scale with using a MySQL database table acting as a queue and some scripts querying the table and doing work. I have worked with maybe a dozen queuing technogies (including super expensive cloud Amazon SQS) and unless your app needs sub second latency on processing messages at greater than 5k per second I just don't see why using these sophisticated systems have benefits? Hope this comment doesn't come across cynical or dismissive. I love seeing new tech like this come out and I am not saying that there aren't use cases I am sure there (maybe HFT?) but curious if anyone has case studies to share of legit queue systems that couldn't scale.
- chasd00 3y ago
  You're right but "MySQL database table acting as a queue" doesn't stand out on resumes/CVs. Plus, new and shiny things attract devs like moths to a flame.
ensocode 3y ago
Very interesting, but looking up the supported standards it just uses TCP. There is no support for AMQP or MQTT. How is interoperability with just some client libraries for Python, Java, ...
- kitd 3y ago
  AMQP & MQTT are messaging protocols. You would need protocol handlers either end with this in the middle. Having said that, you would need to match the protocol to the interaction style. Not all are appropriate.
zoobab 3y ago
Is BlazingMQ faster than ZeroMQ?
- wallstprog 3y ago
  Unlikely, but they seem to be different things altogether. BlazingMQ appears to be a traditional message queue (think ActiveMQ), with message peristence. ZeroMQ is more of a network middleware (think Tibco Rendezvous), and does not include persistence. BlazingMQ also appears to be more of a "platform" or "service" that an app can use (sort of like Oracle, say) -- ZeroMQ includes libraries that one can use to build an app, service or platform, but none is provided "out of the box". Which makes it harder to get started with ZeroMQ, since by definition every ZeroMQ app is essentially built "from scratch". If you're interested in ZeroMQ, you may want to check out OZ (https://github.com/nyfix/OZ https://github.com/nyfix/OZ), which is a Rendezvous-like platform that uses the OpenMAMA API (https://github.com/finos/OpenMAMA https://github.com/finos/OpenMAMA) and ZeroMQ (https://github.com/zeromq/libzmq https://github.com/zeromq/libzmq) transport to provide a full-featured network middleware implementation. OZ has been used in our shop since 2020 handling approx 50MM high-value messages per day on our global FIX network.
qwerty456127 3y ago
Java and C++ clients only?
- callbacker 3y ago
  Python client library is coming in a few weeks.
  - qwerty456127 3y ago
    Great. I also hope .Net. Erlang (Elixir) would be also very useful. Hard to doubt many would appreciate Node.JS. I would also cheer Common Lisp, Guile and Lazarus but I understand these probably are too exotic for Bloomberg to invest in. Whatever, the common denominator is - clients for all sorts of runtimes are expected from such a products. In fact I was very surprised to see such a small client libraries list. Perhaps this is going to get fixed later.
    - callbacker 3y ago
      Hi, internally, we have support for JS but through an enterprise layer which cannot be open sourced. The focus so far has been to prioritize client libraries in languages which are dominant internally. I am sure we will see libraries in other languages due to demand as well as external contributions!
      - qwerty456127 3y ago
        Makes sense. Thank you for doing an amazing job and releasing the code!
        callbacker 3y ago
        Thank you!
the-alchemist 3y ago
Bloomberg has some interesting code! https://github.com/orgs/bloomberg/repositories https://github.com/orgs/bloomberg/repositories Has anyone used comdb2? It seems like an awesome DB that doesn't seem to get the level of love it should.
- cmacleod4 3y ago
  Comdb2 is very heavily used inside Bloomberg; outside not so much.
  - Alifatisk 3y ago
    Is it worth exploring?
  - the-alchemist 3y ago
    If it's heavily used inside Bloomberg, that's good enough for me! BTW, the original Comdb2 paper from 2016 is good [0]. It has features that even today seem awesome. It sounds like a really nice distributed MySQL/PostgreSQL. [0]: http://www.vldb.org/pvldb/vol9/p1377-scotti.pdf http://www.vldb.org/pvldb/vol9/p1377-scotti.pdf
epinephrinios 3y ago
I worked on this team for 3 years along the OGs behind BMQ, specifically on a closed-source middleware with very similar architecture. I am very happy that they finally open sourced one of their main projects! They have very strict standards which are necessary given that C++ really is a minefield. We did a great job avoiding UB by following those coding standards. There is some verbosity that can be avoided but hey these are details. Personally, I have moved to Rust since then and I am not looking back.
- andrewgazelka 3y ago
  Is there something like BMQ but in Rust? Seems interesting you mentioned it.
stolsvik 3y ago
I find it curious that lots of people always drag in Kafka as comparison to message brokers. Their own architecture (log of messages, vs. delivery of messages), but in particular the system architecture they lead to when used (event sourcing, vs. "react to events and commands"), are extremely dissimilar. Kafka has its uses, in particular for massive influx of events, e.g. in a large IoT system - I'd say the perfect example would be continuous measurements of tens of thousands of gauges and sensors on an oil rig or any other large production system, or e.g. the energy meters in every home. But I would personally never use such a system as the inter-service communication layer for a multi-service architecture. It is WAY to heavy coupling. Event sourcing looks fantastic on the surface, but is a disaster for a decades-living system with tons of developers. REST/gRPC is better, but async messaging really rocks.
- marcelius 3y ago
  How is Kafka causing heavy coupling? Serious question.
  - stolsvik 3y ago
    If you use Kafka in an event sourced fashion, whereby each service emits events that it puts on a bunch of Kafka topics, and each service that needs that information consumes those events (by reading and applying those events to its local understanding of the world), I argue that you have effectively made a massive distributed and shared database - shared between all the services. This goes smack in the face of the idea of "one service, one database" 1), where there is a distinct boundary where each service owns their own data. How a given service stores it data is of no concerns to any other service - the communication between them is done using e.g. REST or messages, with a clear contract. Event sourcing / Kafka architectures are the exact opposite of a clear contract. You are exposing the absolute, deep-down innards of the storage solution of a service, by putting its microscopic events down for all to see, and all to consume. You may do aggregations, and emitting more coarse-grained "events" or state changes, thus kinda also exposing "views" of these inner tables, and maybe use a naming scheme to sort of which are "public" and which are not. In the beginning, I really did find the concept of event sourcing to be extremely intriguing: Both the "you can get back to any point in history by replaying up-to", and "forget the databases, just emit your state changes as events" (I truly "hate" databases, as they are the embodiment of state, which is the single one thing that makes my field hard!), the ability to throw up a new projection for anything you'd need (a new "internal view") of the state, and that unit testing could be really smooth. I upon deeper delving into the concepts found that the totality of such a system must quite quickly become extremely heavy: The amount of events, thus needing snapshots. Evolution of events, where you might have no idea of who consumes them (that is, the "shared database" problem). The necessary understanding, throwing a half-century or more of relational databases under the bus. The performance tweaking if things start to lag. Etc etc etc. It would become a distributed system in the worst way a distributed system could be distributed: All state laid out in minute details, little abstractions, and a massive diverse set of different implementations in the different services to get back to a relational view of the data. And this is even before the system gets a decade old, with lots of different developers coming and going, all of them needing to get up to speed, and the total system needing extremely good and hard steering to not devolve into absolute utter chaos. That Kafka can be employed and viewed as a database has been argued hard by Confluent. Here's their former DevEx leader Tim Berglund explaining how databases are like onions, and that in the base of every database there is a commit log. And that this log is equivalent to a set of Kafka topics. 2) So why not just implement all the other layers of the database in application logic? Confluent even have a lot of libraries and whatnots to enable such architectures. 1) "Microservice Architectures", patterns: https://microservices.io/patterns/data/database-per-service.html https://microservices.io/patterns/data/database-per-service.... 2) JavaZone Talk from 2018 by Tim@Confluent: https://2018.javazone.no/program/3a9644e6-15b5-4c66-a28c-c35dc4b85c87 https://2018.javazone.no/program/3a9644e6-15b5-4c66-a28c-c35...
stolsvik 3y ago
I'm the developer of the Message-Oriented Async RPC library Mats3: https://mats3.io/ https://mats3.io/ Its current sole implementation is based on Java Messaging Service JMS API, and it is used in production for a rather large UCITS unit holder registry on Apace ActiveMQ, and all tests runs fine on Apache Artemis MQ. Every time a new message broker comes along, I sit up in the chair and wonder a) about their performance (!), and b) whether they have a JMS client implementation, and c) whether Mats3 works with that! When I tested RabbitMQ's JMS client library, I sadly found that there was rather many differences - things I thought was screamingly obvious, was not available. E.g. as basic function as redelivery: "Normal" MQs try to redeliver N times, typically with a backoff between each attempt, and then, when all N attempts fail, it puts the message on the DLQ. Rabbit instead tight-loops the delivery attempts until either the message goes through, or the sun burns out. To be able to use Rabbit, I would have to use the native API, and implement redelivery and DLQing myself, client side. Also, transactions. I now wonder whether I should make a lower-level abstraction, so that the JMS implementation is converted to a "base" impl, and then the JMS, Rabbit, Bloomberg, NATS, ZeroMQ, Aeron, etc implementations was extensions, or "plugins", or "drivers", to that.
- the-alchemist 3y ago
  Sounds great, and you have lots of nice documentation on the page, but could you provide a TLDR? There's a lot of competition in this area: GRPC, Cap'n'proto (was posted on HN a day or two ago), NATS, etc. I'm also having trouble figuring out if Mats3 is a library (with a JMS API) over a variety of messaging systems (WebSockets, NATS, etc.)? P.S. Some diagrams like https://bloomberg.github.io/blazingmq/ https://bloomberg.github.io/blazingmq/ would be very helpful, especially at https://mats3.io/background/what-is-mats/ https://mats3.io/background/what-is-mats/. If a picture's worth a thousand words, and an animation must be worth at least 10k words. :)
  - stolsvik 3y ago
    I am truly finding it hard to explain it. I have tried along a dozen angles. I actually think it is a completely new concept - and that might be the problem. But there is actual value here, because it "merges" the amazingness of using messaging, with the simplicity of "linear thought" that synchronous, blocking code gives (i.e. ISC using e.g. REST or gRPC). #) There is an illustration on the front-page: https://mats3.io/ https://mats3.io/ This page tries to directly explain the idea - but I guess it assumes that the reader is already extremely familiar with message queuing? https://mats3.io/docs/message-oriented-rpc/ https://mats3.io/docs/message-oriented-rpc/ Here's a set of small answers, from different angles, to "What is Mats?": https://github.com/centiservice/mats3/blob/main/README.md#what-is-mats https://github.com/centiservice/mats3/blob/main/README.md#wh... Here's a way to code up Mats3 endpoints using JBang and a small toolkit which makes it extremely simple to explore the ideas - the article should also be skimmable even without a command line available: https://mats3.io/explore/jbang-mats/ https://mats3.io/explore/jbang-mats/ If you read these and then get it, I would be extremely happy if you gave me a sentence or paragraph that would have led you to understanding faster! The concept of messaging with queues and topics are essential to Mats3 - but the use of JMS is really just a transport. I could really have used anything, incl. any MQ over any protocol, or ZeroMQ, or plain TCP - or just a shared table in a database (but would then have had to implement the queue and topic semantics). As a developer, you code Mats3 Endpoints by using the Mats3 API, and you initiate Mats3 Flows using the API. You need an implementation of the MatsFactory to do this, and the sole existing is the JmsMatsFactory - which works on top of ActiveMQ and Artemis's JMS clients. Wrt. WebSockets, that is a transport typically between a server, and a end-user client, e.g. an iOS App. Actually, there's also a "sister project", MatsSockets, that bring the utter async-ness of Mats3 all the way out to the client, e.g. a webpage or an app. https://matssocket.io/ https://matssocket.io/ NATS is, AFAIU, just a message broker, with some ability to orchestrate. Fundamentally, I do not like this concept of orchestration as an external service - this is one of the founding ideas of Mats3: Do the orchestration within each service, as you would do if you employed REST as the ISC mechanism. I do however assume that one could make a Mats3 implementation on top of NATS client libs. #) Java's Project Loom is extremely interesting to me, as its argument for using threads as the basis of concurrency instead of asynchronous styles of programming, is exactly the same rationale for which I made Mats3: It is much simpler for the brain to grok a linear, sequential, "straight-down" code, than lots of pieces of code that are strung together in a callback-hell. Async/await is a way to make such callback-hell more readable, "linearaize it" (but you still have the "colored methods" problem which Loom just obliterates in a perfect explosion). One could argue that this is what I have achieved with Mats3 when using messaging.

695 points by carride 3y ago

What a phenomenal landing page. Thanks to the animations I have a good idea of what this does and how it does it. You barely need to know what message queuing is to understand it.

ElongatedMusket 3y ago
Getting started pages are great too, with guided examples and use cases: https://bloomberg.github.io/blazingmq/getting_started https://bloomberg.github.io/blazingmq/getting_started BMQ license is apache 2.0 for anyone curious
scottlamb 3y ago
The animations are very slick, but I can still grumpily pick at them: the "Clustering and Quorum-based Replication" one shows 4 total servers (suboptimal number because quorum requires 3/4 where I'd prefer to require 2/3 or 3/5) and all the followers acking before the primary proceeds (the animation could show one lagging instead).
- amelius 3y ago
  Also, why is the middle replica special, in that the message is passed through it in the end?
- BeeOnRope 3y ago
  4 might be useless compared to 3 from a "required quorum" PoV but if you want to stay up with 1 failure and the load can't be handled on 3 servers but 4 is OK, it is optimal isn't it? I.e. selection of ideal server count is a multi-dimensional problem.
  - scottlamb 3y ago
    Worse than useless from a quorum perspective, because you've increased the number of servers that have to participate in quorum (quickly) without increasing the number that don't. Looking at it the other way, you go from having problems when 2/3 are slow/broken to having problems when 2/4 are slow/broken. (also note slow doesn't even mean the machine is having problems; in geographically distributed setups it may just be due to physical distance from the leader.) Yes, 4 servers might handle more load than 3 servers [1], but so would 5 servers. I'd guess that if you've got enough message queue traffic that 3 servers can't handle the load, those servers aren't a significant fraction of your total cost. You might as well go to 5. Likewise, if 5 aren't enough, I'd skip over 6 and go to 7. If 7 aren't enough...I'd probably rethink my design entirely...but I'd prefer 9 to 8. Alternatively, quorum systems can support "zero-weight replicas" which effectively don't vote but still can be useful for eventually consistent reads. Then you have this server that helps handle the load but doesn't increase the number of servers that have to participate in a round. [1] if the load is overwhelmingly eventually consistent read operations and their multi-hop topology with proxies doesn't sufficiently reduce the load on the replicas. It's not obvious to me if this scenario is plausible or not.
gr33nq 3y ago
Does anyone recommend any particular piece of software for creating these types of visualizations with relative ease? I've always been a fan of animations for helping to understand some technical topics.
- erwinkle 3y ago
  They use anime.js for their animations https://animejs.com/ https://animejs.com/ As seen in their source code here: view-source:https://bloomberg.github.io/blazingmq/assets/animations/animations/bmq/message-paths/index.html https://bloomberg.github.io/blazingmq/assets/animations/anim...
- maxwellito 3y ago
  As erwinkle mentioned, these animations use anime.js If you want to build something but don't really know how to do it, I would suggest the following: 1. Build the visual content in an SVG with Inkscape. It should be easy for you to build the layout with assets/item to animate. Don't forget to group items and set names so you can reference them later. 2. Once exported, insert the SVG in an HTML page and add anime.js. This library will let you animate the content you just created (you should be able to reference your named items from Inkscape and animate them). The learning curve might be tough depending on your experience regarding animation (or CSS in general). Good luck :)
- treprinum 3y ago
  animatron.com for animated SVG or JS. I used it for some algorithm animations myself.
- danielvaughn 3y ago
  I've heard Framer can make some impressive animations, but I don't know much about how well they export to web. What's on display here is really cool, it's all animated SVG.
- jasonlotito 3y ago
  Okay, this is going to sound weird, but I've used Keynote in the past. I'm sure PowerPoint can do this as well, but Keynote has Magic Move, and you can export this animation. It's very easy to work with, and do simple animations. I'm sure there are much better tools for this, but as someone who knew Keynote, this was always effective for me.
rewmie 3y ago
I agree, this is perhaps the very best landing page I ever saw. What a treat.
nine_zeros 3y ago
Agreed. This feels like a page created from enthusiasm, not from a place of promo-seeking performance review.

hkt 3y ago

It has a gorgeous landing page but I'm not really sure why this is better than any other MQ. Can someone more knowledgeable provide any insight into its comparative advantages? Edit: I did see the comparisons page, but.. well, there's more to life than Rabbit and Kafka!

traceroute66 3y ago
> Can someone more knowledgeable provide any insight into its comparative advantages? Well, you know, there is the "Comparison With Alternative Systems" page[1]. :-) Sadly it doesn't include NATS in the comparison. I would have been interested to see that. But the usual suspects (Rabbit & Kafka) are given a mention. [1] https://bloomberg.github.io/blazingmq/docs/introduction/comparison/ https://bloomberg.github.io/blazingmq/docs/introduction/comp...
- rlonstein 3y ago
  I'd also be interested in seeing how it compares to AMQP implementations (Apache Qpid for example) and any of the proprietary message queue systems (IBM and Microsoft) they refer to but don't name-check.
vyskocilm 3y ago
FJI: There is a faq entry about that https://bloomberg.github.io/blazingmq/docs/introduction/comparison/ https://bloomberg.github.io/blazingmq/docs/introduction/comp...
Cthulhu_ 3y ago
I mean one unsubstantiated armchair take: it's newer than Rabbit / Kafka (RabbitMQ was built in 2007, Kafka in 2011), since then the learnings from distributed systems, message queues and programming languages has improved by leaps and bounds, and the existing ones will have backwards compatibility / legacy issues. That said, there are indeed plenty of commercial and noncommercial alternatives, so now that it's been published I'm sure there will be some thorough people making comparisons soon!
MathMonkeyMan 3y ago

mgaunard 3y ago

I don't understand how you can make queueing high-performance. Queueing things is literally the opposite of making things fast.

LoganDark 3y ago
Queueing things can result in better throughput because you don't have to wait for the other side to process your last message before you can begin work on the next one.
- jeffbee 3y ago
  That's just buffering. A lot of people use Kafka this way, as an impedance adapter for a producer with an internal architecture that can't tolerate blocking on production. Of course this requires you to unrealistically assume that Kafka is a zero-impedance message sink. But what I think the other post is alluding to is the fact that in-order delivery is often not a requirement, and can be an anti-feature. I know that in every use of Kafka I have personally encountered the relative order of messages was quite irrelevant but the in-order implementation of Kafka converts the inability to process a single record into an emergency.
  - therealdrag0 3y ago
    There’s a lot of brokers without ordering guarantees. ActiveMQ for example.
hbarka 3y ago
1,000,000 messages per second isn’t high performance?
- mgaunard 3y ago
  A message is typically 1k. That would mean 1GB/s. That is decent but still below the speed of the network, which should be the main blocker.
3y ago

fidotron 3y ago

That fan-out functionality is particularly interesting for anyone needing to implement the upstream part of a firehose API, which of course Bloomberg do.

arnold_palmur 3y ago

I wonder how this compares to something like Aeron performance-wise - though Aeron is strictly messaging (not queueing).

fizwhiz 3y ago

> This section explains the leader election algorithm at a high level. It is by no means exhaustive and deliberately avoids any formal specification or proof. Readers looking for an exhaustive explanation should refer to the Raft paper, which acts as a strong inspiration for BlazingMQ’s leader election algorithm. So their own homegrown leader election algorithm? > BlazingMQ’s leader election and state machine replication differs from that of Raft in one way: in Rafts leader election, only the node having the most up-to-date log can become the leader. If a follower receives an election proposal from a node with stale view of the log, it will not support it. This ensures that the elected leader has up-to-date messages in the replicated stream, and simply needs to sync up any followers which are not up to date. A good thing about this choice is that messages always flow from leader to follower nodes. > BlazingMQ’s elector implementation relaxes this requirement. Any node in the cluster can become a leader, irrespective of its position in the log. This adds additional complexity in that a new leader needs to synchronize its state with the followers and that a follower node may need to send messages to the new leader if the latter is not up to date. However, this deviation from Raft and the custom synchronization protocol comes in handy because it allows BlazingMQ to avoid flushing (fsync) every message to disk. Readers familiar with Apache Kafka internals will see similarities between the two systems here. "a new leader needs to synchronize its state with the followers and that a follower node may need to send messages to the new leader if the latter is not up to date". I thought a hallmark of HA systems was fast failover? If I come to your house and knock on the door, but it takes you 10mins to get off the couch to open the door, it's perfectly acceptable for me to claim you were "unavailable". Pedants will argue the opposite.

anentropic 3y ago
"deliberately avoids any formal specification or proof" how is this possibly a selling point?
- scottlamb 3y ago
  > "deliberately avoids any formal specification or proof" > how is this possibly a selling point? Context. In an overview doc? The informal version is accessible to more audiences. As the canonical design doc for the system? It's certainly not.

i-use-nixos-btw 3y ago

Off topic here but > Carefully architected and written in C++ from the ground up with no dependency on any external framework, BlazingMQ provides… What a weird selling point. There are pros and cons to having no dependencies. Not long ago, it was a common decision for C++ projects because dependency management was a mess. But that hasn’t been the case for a long time (arguably we have the opposite problem - there are so many ways of doing it: Conan, vcpkg, bazel, spack, raw cmake, nix, etc) So what would the pros and cons be today and why is it a selling point? For instance, a pro is that everything is bespoke to the task at hand, no hammering square pegs into round holes designed for a slightly different use case. A con is that the entire attack surface is managed by one team. CVEs identified and solved on another project is a pretty good thing when you depend on that project. The main reason I’m surprised, though, is that there are some no-brainer dependencies these days. Fmt, catch2/gtest, metaprogramming libraries, etc.

omeid2 3y ago
Sending patches to fix a project is hard work, sending patches to fix issues in a projects' upstream projects is even harder. This alone is an important point to avoid dependencies, specially if you're anything like Bloomberg.
- i-use-nixos-btw 3y ago
  I run into that problem a lot, to be fair. Though my approach is to fork and maintain the fork until my PR is accepted, merged, and in a tagged release. Another is to apply a diff to the dependency within the build system, which is what I do a lot in Nix (mainly to solve cmake issues on hermetic builds). Either way, it isn't unsurmountable. It doesn't really matter so much if it's used as an executable rather than as a library, though in the case of the latter it's really handy to be able to e.g. pass a custom spdlog sink for logging.
deleted 3y ago
[deleted]
callbacker 3y ago
Hi, one of the authors here. BlazingMQ depends on two other open source C++ libraries: https://github.com/bloomberg/bde https://github.com/bloomberg/bde and https://github.com/bloomberg/ntf-core https://github.com/bloomberg/ntf-core. I believe documentation writer wanted to highlight that BlazingMQ does not depend on frameworks like ZooKeeper, etc.

notjoemama 3y ago

It doesn't use UDP. So my tongue-in-cheek reply is, it doesn't qualify to have "blazing" in the name. Hopefully that's not just funny inside my own head. :) https://bloomberg.github.io/blazingmq/docs/architecture/network_transport/ https://bloomberg.github.io/blazingmq/docs/architecture/netw...

rewmie 3y ago
> It doesn't use UDP. So my tongue-in-cheek reply is, it doesn't qualify to have "blazing" in the name. Arguably, if you start with UDP but you also need to ensure retransmission and reordering, you eventually end up reimplementing TCP but poorly.
- jayd16 3y ago
  I guess you could implement QUIC instead.
- lordnacho 3y ago
  Check out Aeron, they have a way to multicast (thus UDP) reliably and ordered. Based on negative acks. Super fast.
  - HappySweeney 3y ago
    NORM from the Naval Research Labs is another implementation of the same concept. https://github.com/USNavalResearchLaboratory/norm https://github.com/USNavalResearchLaboratory/norm
  - jonathanoliver 3y ago
    Aeron is another awesome project from Martin Thompson (Disruptor, Simple Binary Encoding, Mechanical Sympathy, etc.). That guy really knows his stuff regarding performance.
  - phishhook 3y ago
    Sounds like PGM

say_it_as_it_is 3y ago

Why is Bloomberg approving the open sourcing of this system?

esafak 3y ago
They must not consider it a trade secret, so they can earn good will, generate public contributions, and advertise their brand by sharing it.
SkipperCat 3y ago
If it gains wide adoption, then they've got a global community of people providing free software engineering for their messaging platform. Its a win-win.
loire280 3y ago
Why wouldn’t they? This technology is not a competitive advantage for their company, wouldn’t make sense for them to sell as a standalone product, and it helps their engineering department attract good talent.
BossingAround 3y ago
Hiring strategy--there are a ton of people who'll take a slightly lower salary if they can get a public record of their code commits.
Cthulhu_ 3y ago
Because they're not making money off of their queueing system, but what they build with it; to them it's a cog in their machine, and open sourcing it means others may adopt it and help them maintain and improve it.
puchatek 3y ago
Same reasons any company open sources their software
sdesol 3y ago
Here's some community stats for Bloomberg open source projects https://devboard.gitsense.com/bloomberg https://devboard.gitsense.com/bloomberg For the 200 repos that they have made open source, they had over 6000 people contribute feedback and/or code. So in addition to attracting talent, they also get more testers and feedback. Full Disclosure: This is my tool

insanitybit 3y ago

> Supports a unique multi-hop network topology leading to a distribution tree for each queue, thereby leading to network bandwidth savings for certain use cases. This is very interesting. One of the benefits of Kafka is partition routing so I'm curious to learn how this may compare.

Sparkyte 3y ago
No idea, but I am interested too. One of the reasons this was probably created was because of a network constrained environment where data is expensive in more than one way. Cloud Providers can be expensive.
- eric-hu 3y ago
  > Cloud Providers can be expensive. Do you have a story around this? If so, would love to hear more about it.
  - lelandbatey 3y ago
    Specifically, egress from Cloud Providers (transmitting data from inside the cloud to a computer outside the cloud) is the usual culprit for being "very expensive". It's part of a pricing strategy by cloud providers that encourages folks to put all their infrastructure inside of the cloud and to have none of it outside the cloud. As one example of the ink spilled over this topic, see this discussion from 2 years ago about AWS's very high egress fees: Article "AWS’s Egregious Egress" (2021) https://blog.cloudflare.com/aws-egregious-egress/ https://blog.cloudflare.com/aws-egregious-egress/ HN discussion: https://news.ycombinator.com/item?id=27930151 https://news.ycombinator.com/item?id=27930151
    - beyonddream 3y ago
      Ingress can also be costly especially if there is a steady state of high load traffic sent from on-prem machines to machines inside cloud (not sure about aws but have experienced this with azure where we had to resort to buy their expressroute which was very costly and ultimately unsustainable for us)

davidkunz 3y ago

It's called BlazingMQ but not written in Rust?!

reactordev 3y ago
I’ll admit, I went into their repo expecting to find Rust, or at the very least go. Pleasantly surprised to see C++ code though I would need to look further to see if it’s “good” C++ code. The naming conventions of the source tree lends me to suspect it’s not.
- deleted 3y ago
  [deleted]
- beanjuiceII 3y ago
  i too determine if code is good based off its naming conventions and other things like my linter is set to enforce MiXeD_SpOnGeBoB-KeBaB_CaSe as the default casing
  - gabereiser 3y ago
    Better than p_f_var_t;
Cthulhu_ 3y ago
The language is an implementation detail; what makes you think Rust would be better suited for such a specialist piece of software?
- higherhalf 3y ago
  It's a joke, as some Rust projects used to repetitively claim to be "memory safe & blazing fast", thus becoming a tongue-in-cheek phrase. I also anticipated language to be Rust, but it would be way too comical. https://github.com/mTvare6/hello-world.rs https://github.com/mTvare6/hello-world.rs

29athrowaway 3y ago

420 messages per second.

rurtrack 3y ago
Sweet
- 29athrowaway 3y ago
  Dude
  - deleted 3y ago
    [deleted]

endisneigh 3y ago

it never ceases to amaze me how the software engineering profession has very smart folks reimplement the same conceptual primitives over and over again. I'm curious if hospitals (if that's even analogous to a company) do similar things by having different processes to solve the same problem. I imagine this is used for Bloomberg (the terminal) and not Bloomberg (the website)? going back to the article - fantastic animations. I'm just as curious to how that was made as the queue itself.

BossingAround 3y ago
> I'm curious if hospitals (if that's even analogous to a company) do similar things by having different processes to solve the same problem. Of course they do, unless the processes are regulated by a country (such as handling highly addictive opiates in the US).
- quercusa 3y ago
  In the US, all hospitals are required to use electronic health records (EHRs) and most are implemented in Epic, the leading EHR system. But they are almost always electronic implementations of their old paper record systems, dating back to who-knows-when.
- thfuran 3y ago
  Yeah, pretty much any medium to large company is going to have a bunch of processes for doing things that any company that size needs to do and some more that are a bit more industry/domain specific but still basically the same as those at every other company in the space.
Sparkyte 3y ago
Part of this evolution is from funded development to opensource development. Then there is language variances like flavors so what ends up happening is a million different conceptual primitives that fundamentally the same. It isn't the same because of a few things. At the end of the day too it is also about how good the support is over the rest. If a new MQ can demonstrate better tooling nothing stops a company from adopting it.

mkl95 3y ago

How does it compare to NATS.io?

leetharris 3y ago
It is amazing to me that NATS.io is not more popular. It is so much better than Kafka, extremely simple, and extremely light.
- slantedview 3y ago
  Nats is simple because it doesn't bother addressing some of the scalability problems that Kafka was designed to address, like clustering with partitions: https://docs.nats.io/legacy/stan/intro/partitioning https://docs.nats.io/legacy/stan/intro/partitioning
  - coder543 3y ago
    You linked to NATS Streaming, which has been deprecated for years. It’s not relevant. It can still be used for legacy code that still needs it, as the URL even shows, but it is not recommended for new applications. The current standard is NATS JetStream, which absolutely addresses scalability, arguably to an even larger degree than Kafka, since it can be used to create a globally interconnected supercluster of clusters where messages can be configured to flow between regions as needed. I doubt anyone would run a Kafka cluster that spans multiple regions.
    - kitd 3y ago
      I doubt anyone would run a Kafka cluster that spans multiple regions. Stretch clusters and clusters with brokers in different AZs are absolutely a thing in Kafka. I make no comment on the ease relative to Nats, which I don't know at all.
      - coder543 3y ago
        I was talking about different regions altogether, not just different AZs. My understanding is that it’s generally a bad idea in Kafka due to the latency, but I could be wrong. EDIT: Definitely sounds like there are some tight latency requirements[0]: > A stretched 3-data center cluster architecture involves three data centers that are connected by a low latency (sub-100ms) and stable (very tight p99s) network, usually a “dark fiber” network that is owned or leased privately by the company. 100ms would theoretically be enough to span the contiguous United States, but the references to "very tight p99s" and "dark fiber" make me wonder if 100ms is actually acceptable, or just a theoretical maximum that can only be allowed under absolutely perfect conditions. Either way, suggesting the user should have access to dark fiber between their regions does not fill me with confidence about the robustness of this solution. I'm sure it would be fine for geographically close datacenters, as AZs are designed to be. [0]: https://docs.confluent.io/platform/current/multi-dc-deployments/multi-region-architectures.html#stretched-cluster-3-data-center https://docs.confluent.io/platform/current/multi-dc-deployme...

mizzlr_ 3y ago

LavinMQ is much better.

konart 3y ago
Better how?
renewiltord 3y ago
Appears 10^3 times latency for 1 KiB message according to here but better at lower 16 B size https://www.cloudamqp.com/blog/lavinmq-benchmarks.html https://www.cloudamqp.com/blog/lavinmq-benchmarks.html Seems situation dependent.
- phishhook 3y ago
  Does that do HA/replication/clustering etc though?

Thaxll 3y ago

Which version of C++ is this built on?

coder543 3y ago
One person opined that it is similar to C++03: https://news.ycombinator.com/item?id=36898899 https://news.ycombinator.com/item?id=36898899
hallfox 3y ago
Hi, dev here. BlazingMQ is written with a baseline of C++03, though our docker tutorial builds it with C++17.
frutiger 3y ago
It targets AIX and Solaris (in addition to Linux) and vendor-supported C++ toolchains have spotty support for C++11 and newer.

user6723 3y ago

[flagged]

Cthulhu_ 3y ago
I don't understand the language preference; it's an implementation detail.

synergy20 3y ago

how does this compare against other existing message queuing systems? the github shows its codebase is in c++ with gcc-7.3, that means it might be using c++14 in the code, or even c++17, which is nice.

eska 3y ago

I checked the docs, but couldn’t find anything about mqtt compat?

m00x 3y ago
I don't think this is an MQTT broker.

renewiltord 3y ago

Very interesting. Thank you for benchmarks. Thank you for sharing. In a zero-replication no-leader-election (or all-in-one-node) scenario what is min latency observed? It appears predominant delay is in ACKing to ensure durability, right? C++ and Java client examples are very simple which is cool. Would appreciate if there is hook to enable `SO_TIMESTAMPNS` on socket and retrieve `cmsg` to get accurate timestamping and permit client code benchmarking.

callbacker 3y ago
Hi, one of the authors here. > In a zero-replication no-leader-election (or all-in-one-node) scenario what is min latency observed? We saw sub-millisecond latency in this case (typically double-digit microseconds).
- renewiltord 3y ago
  Splendid! Great work, guys. And thanks again for sharing. Dockerfile is good way to share build deps.
  - callbacker 3y ago
    Thank you!

1letterunixname 3y ago

Marketing is all well and good. Let's see the data of p99 latency and throughput numbers compared to rabbitmq, kafka, activemq, qpid, and hornetq. Also, it's worth identifying and setting aside use-cases for broker-less messaging where 0mq, nanomsg, etc. could be more appropriate. Edit: There's only published performance data of itself, which has little value: https://bloomberg.github.io/blazingmq/docs/performance/benchmarks/ https://bloomberg.github.io/blazingmq/docs/performance/bench...

glonq 3y ago

Are they working on other API's besides Java and C++?

frakkingcylons 3y ago
The python client library is planned to be published. They say they need to remove some dependencies and update the workflow to something more suitable for OSS.
callbacker 3y ago
Hi, one of the authors here. We will be releasing a Python client library in a few weeks.
- KrishnaAnaril 3y ago
  Any plans to support .NET?
  - mikece 3y ago
    There are a number of ways to wrap native code libraries as a managed library. I did it once (packaged an Obj-C library as a .NET DLL for a Xamarin app) back in 2018 but had lots of help from a teammate who knew that dance far better than me and I forgot all of the details. I googled and found at least four different blogs posts showing how to do it. I know that's not an answer but if you're feeling adventurous you might be able to help the project.
    - glonq 3y ago
      +1 vote from me for a first-class client for .Net core. I'd tolerate wrappers for certain things, but not for this thing.
  - callbacker 3y ago
    Hi, no plans as of yet. Please also see my response to the Golang client library request above.

alberth 3y ago

> “Carefully architected and written in C++” Amazing to see a non-Rust product posted to HN.

geertj 3y ago
I’ve been getting pretty deep in C++ recently and the name Bloomberg shows up all across the C++ standards community. They seem to be a strong C++ shop.
- tkhattra 3y ago
  yep, the authors of "Embracing Modern C++ Safely" [1] are Bloomberg folks [1] https://vittorioromeo.info/emcpps.html https://vittorioromeo.info/emcpps.html
qwertox 3y ago
True, but maybe its age could play some role in it being a C++ project. > BlazingMQ is an actively developed project and has been battle-tested in production at Bloomberg for 8+ years. https://github.com/bloomberg/blazingmq https://github.com/bloomberg/blazingmq
harerazer 3y ago
Recently, mold 2.0 was posted, which is also C++.
beanjuiceII 3y ago
don't worry the army of Rust authoritarians are waiting to reply as we speak. how software is just not ethical or correct unless its in rust

tetrep 3y ago

Is this implemented with any sort of thought to security? I don't see anything that implies as such, all FAQ and comparisons to other message queues say nothing of security (I skimmed and ctrl+f so I could have missed it). This doesn't bode well for a network-based application written in a memory unsafe language. edit: I'm surprised so many people are interpeting this as a weird cargo cult thing. Application security includes a lot of mundane and critical things like: Authentication, authorization, message signing / authentication (e.g. HMAC), encryption, secret/key management, how the system handles updates, etc. You can read a bit more about it here: https://en.m.wikipedia.org/wiki/Application_security https://en.m.wikipedia.org/wiki/Application_security

mrits 3y ago
Would you prefer it written in Rust with everything wrapped in unsafe block?
- tetrep 3y ago
  I care more about methodology than specific methods. I would like applications that handle I/O, especially network I/O, to be designed and built with security in mind. I asked my question because that doesn't appear to be the case here, and I think that is dissapointing and concerning from a security perspective.
- coder543 3y ago
  That is an obviously false dichotomy. The author of the comment never mentioned Rust, and wasn’t asking about Rust. Such a false dichotomy feels like flamebait, which is against HN guidelines. Your false dichotomy also implies that “unsafe” blocks are as unsafe as C++, which is not true. “Unsafe” in Rust turns off very few checks[0], most of them are still active. No one would write a serious Rust program entirely in unsafe anyways. Regardless, asking about security considerations is a valid thing to do, even if it were written in Rust. Security is not just about memory safety. [0]: https://doc.rust-lang.org/std/keyword.unsafe.html#unsafe-abilities https://doc.rust-lang.org/std/keyword.unsafe.html#unsafe-abi...

fadedsignal 3y ago

A yet another MQ :-) I quickly checked the performance page and it looks much better than its competitors. Documentation is better as well.

MathMonkeyMan 3y ago
Last time I checked, they weren't even building with optimization, and it was still the fastest thing in the West. Peeved me, but whatever. "Every byte counts" was the motto of the team that designed it.
- deleted 3y ago
  [deleted]

travisgriggs 3y ago

Interesting that there's a comparison (albeit low on details) to Rabbit, but not MQTT? Anyone had experience with both MQTT and BlazingMQ able to compare/contrast?

dbrueck 3y ago
Isn't MQTT more of a standard than an implementation?
- winrid 3y ago
  Yes. They prob mean rabbitmq.
- smarx007 3y ago
  I would be interested in the differences between BlazingMQ, AMQP, and MQTT protocols and the invariants afforded to the clients, not the implementations per se.
- sheerun 3y ago
  I think he means implementation of MQTT protocol, like https://mosquitto.org/ https://mosquitto.org/

sitkack 3y ago

TLA 0.1% Finally! Thank you!

callbacker 3y ago
Thanks! The spec can be found here: https://github.com/bloomberg/blazingmq/tree/main/etc/tlaplus https://github.com/bloomberg/blazingmq/tree/main/etc/tlaplus

weekendcode 3y ago

Built in C++, awesome :)

sho 3y ago
Is that really your reaction to C++? These days when selecting a critical infrastructure component I'm very inclined to prefer memory-safe by default, like rust or (somewhat less so) golang. Sure, C/C++ projects that have been around and under attack for years (redis, postgres, etc) are fine but they've had those years of battle testing. For a new project I really feel a lot safer if they're built in something more failsafe.
- wiseowise 3y ago
  Can you show me where modern C++ is unsafe and why it matters in this specific context?
  - epinephrinios 3y ago
    Read this - https://docs.google.com/document/d/e/2PACX-1vRZr-HJcYmf2Y76DhewaiJOhRNpjGHCxliAQTBhFxzv1QTae9o8mhBmDl32CRIuaWZLt5kVeH9e9jXv/pub https://docs.google.com/document/d/e/2PACX-1vRZr-HJcYmf2Y76D...

nodesocket 3y ago

Awesome landing page and looks great. Documentation is very well done. Would love to see more clients (python) and a web u/i.

callbacker 3y ago
Hi, one of the authors here. We will be releasing a Python client library in a few weeks.

melenaboija 3y ago

I have been working on an engine to perform heavy computations that is distributed and my messaging choice has been gRPC + Flatbuffers + proxys and I was wondering if someone could name some reasons or use cases to chose message brokers vs a lower level approach like gRPC. Is it mostly because of synchronous/asynchronous needs or there is something else such as latency or even ease of implementation?

ozarker 3y ago
Depends on your use case. Queueing messages enables the competing consumer flow which implicitly allows load-leveling which is great if your traffic is spiky and you need a little time to auto-scale or other scenarios like that. If you’re just looking to blast out data to all consumers or just load-balancing is fine, approaches like yours are probably better.
imachine1980_ 3y ago
grpc client a -> grpc server a for client a grpc client b -> grpc server a for client b grpc client a -> message broker(implemen a) --> grpc client b -> message broker(implemen b) --> message broker-> your app for the broker this make sense if dont make sense manage the quque in your app if ypu want to abstract the implementations is worth if you have n to 1 or n to n and all works different
pravus 3y ago
If your workers can hop onto the next task without having to reach out for any additional state or report back to anything synchronously a distributed log/spool/queue/broker works well. If you need light-weight reporting like a status code you can use RPC/reply mechanisms if the broker supports it (e.g. rabbitmq), build your own (custom headers), or use an API/gRPC call. Everything else starts needing some sort of more sophisticated coordination mechanism and gRPC will look attractive for that although there are usually other options as well.
stolsvik 3y ago
It is the asynchronous and decoupled nature of messaging that pulls architectures towards message brokers. In a "MOM Architecture", the producing of messages, and consuming of messages, is completely decoupled - both in "space" (HA, location transparency, replication etc) and in time (the async nature). This leads to a whole heap of benefits. I've outlined a few of them here, in the "Rationale for Mats3", my Message-Oriented Async RPC library: https://mats3.io/background/rationale-for-mats/ https://mats3.io/background/rationale-for-mats/ You will find that the Reactive Manifesto also lays out the same type of arguments (there's some links to it from the list in the first link): https://www.reactivemanifesto.org/ https://www.reactivemanifesto.org/ and its glossary: https://www.reactivemanifesto.org/glossary https://www.reactivemanifesto.org/glossary

MathMonkeyMan 3y ago

They actually open sourced it! I worked on this team very briefly. Great team working on some interesting tech.

deleted 3y ago
[deleted]
theptip 3y ago
Would love to hear any technical history you can share, what did this replace, and what shortcomings led to it being built? (Seems quite similar to Kafka but perhaps more emphasis on low latency rather than raw throughput?)
- callbacker 3y ago
  Some of your questions might be answered in the accompanying blog - https://www.bloomberg.com/company/stories/bloomberg-publishes-blazingmq-modern-high-performance-message-queuing-system https://www.bloomberg.com/company/stories/bloomberg-publishe..., and the comparison doc https://bloomberg.github.io/blazingmq/docs/introduction/comparison/ https://bloomberg.github.io/blazingmq/docs/introduction/comp....

accrual 3y ago

Does anyone here do large scale interface stuff with many millions of messages? I do this in healthcare and would like to try applying my skills somewhere else...

didip 3y ago
Just about any large tech companies and unicorns have a large queue (or many of them).
matthewpersico 3y ago
https://careers.bloomberg.com/job/search https://careers.bloomberg.com/job/search :-)

Eumenes 3y ago

As much as Bloomberg (the guy and company) creep me out, this is impressive.

midoridensha 3y ago
I have a similar feeling about Facebook, but zstd is amazing.

kajaktum 3y ago

I don't doubt that it is fast but please don't name your thing <claim><thing>.

seabrookmx 3y ago
Not a big FastAPI fan eh? Or MongoDB? What about BigQuery? So many bad names!
wiseowise 3y ago
Why?
- kajaktum 3y ago
  Because the name is a technical claim. Is it fast today? Maybe but will it still be considered "blazing" in 20 years? This is equivalent to naming your thing VeryFastMQ. I can already imagine people saying "Oh yea BlazingMQ is craaaawling right now, I am thinking of replacing it with SuperiorMQ".

favflam 3y ago

From the "leadership election" snippet, this setup looks perilously close to the peer to peer propagation of transactions and blocks in a blockchain system such as Solana. I suppose the latency requirements are much tighter than 400ms though. I know Solana does mutual TLS over QUIC authentication between peers and does bandwidth rate limiting via stake. Also, the topology is self organizing. Is it worth copying this tech into a message queue system?

k2xl 3y ago

Congrats on open sourcing this, but I am curious if anyone here has ever had an actual scale issue that required changing from some "slower" queuing system (like RabbitMQ, Kestrel, Kafka, etc) where you actually needed to change queuing systems? Maybe I'm old school but 99% of the use cases I've worked with in my 20 year career could scale with using a MySQL database table acting as a queue and some scripts querying the table and doing work. I have worked with maybe a dozen queuing technogies (including super expensive cloud Amazon SQS) and unless your app needs sub second latency on processing messages at greater than 5k per second I just don't see why using these sophisticated systems have benefits? Hope this comment doesn't come across cynical or dismissive. I love seeing new tech like this come out and I am not saying that there aren't use cases I am sure there (maybe HFT?) but curious if anyone has case studies to share of legit queue systems that couldn't scale.

chasd00 3y ago
You're right but "MySQL database table acting as a queue" doesn't stand out on resumes/CVs. Plus, new and shiny things attract devs like moths to a flame.

ensocode 3y ago

Very interesting, but looking up the supported standards it just uses TCP. There is no support for AMQP or MQTT. How is interoperability with just some client libraries for Python, Java, ...

kitd 3y ago
AMQP & MQTT are messaging protocols. You would need protocol handlers either end with this in the middle. Having said that, you would need to match the protocol to the interaction style. Not all are appropriate.

zoobab 3y ago

Is BlazingMQ faster than ZeroMQ?

wallstprog 3y ago
Unlikely, but they seem to be different things altogether. BlazingMQ appears to be a traditional message queue (think ActiveMQ), with message peristence. ZeroMQ is more of a network middleware (think Tibco Rendezvous), and does not include persistence. BlazingMQ also appears to be more of a "platform" or "service" that an app can use (sort of like Oracle, say) -- ZeroMQ includes libraries that one can use to build an app, service or platform, but none is provided "out of the box". Which makes it harder to get started with ZeroMQ, since by definition every ZeroMQ app is essentially built "from scratch". If you're interested in ZeroMQ, you may want to check out OZ (https://github.com/nyfix/OZ https://github.com/nyfix/OZ), which is a Rendezvous-like platform that uses the OpenMAMA API (https://github.com/finos/OpenMAMA https://github.com/finos/OpenMAMA) and ZeroMQ (https://github.com/zeromq/libzmq https://github.com/zeromq/libzmq) transport to provide a full-featured network middleware implementation. OZ has been used in our shop since 2020 handling approx 50MM high-value messages per day on our global FIX network.

qwerty456127 3y ago

Java and C++ clients only?

callbacker 3y ago
Python client library is coming in a few weeks.
- qwerty456127 3y ago
  Great. I also hope .Net. Erlang (Elixir) would be also very useful. Hard to doubt many would appreciate Node.JS. I would also cheer Common Lisp, Guile and Lazarus but I understand these probably are too exotic for Bloomberg to invest in. Whatever, the common denominator is - clients for all sorts of runtimes are expected from such a products. In fact I was very surprised to see such a small client libraries list. Perhaps this is going to get fixed later.
  - callbacker 3y ago
    Hi, internally, we have support for JS but through an enterprise layer which cannot be open sourced. The focus so far has been to prioritize client libraries in languages which are dominant internally. I am sure we will see libraries in other languages due to demand as well as external contributions!
    - qwerty456127 3y ago
      Makes sense. Thank you for doing an amazing job and releasing the code!
      - callbacker 3y ago
        Thank you!

the-alchemist 3y ago

Bloomberg has some interesting code! https://github.com/orgs/bloomberg/repositories https://github.com/orgs/bloomberg/repositories Has anyone used comdb2? It seems like an awesome DB that doesn't seem to get the level of love it should.

cmacleod4 3y ago
Comdb2 is very heavily used inside Bloomberg; outside not so much.
- Alifatisk 3y ago
  Is it worth exploring?
- the-alchemist 3y ago
  If it's heavily used inside Bloomberg, that's good enough for me! BTW, the original Comdb2 paper from 2016 is good [0]. It has features that even today seem awesome. It sounds like a really nice distributed MySQL/PostgreSQL. [0]: http://www.vldb.org/pvldb/vol9/p1377-scotti.pdf http://www.vldb.org/pvldb/vol9/p1377-scotti.pdf

epinephrinios 3y ago

I worked on this team for 3 years along the OGs behind BMQ, specifically on a closed-source middleware with very similar architecture. I am very happy that they finally open sourced one of their main projects! They have very strict standards which are necessary given that C++ really is a minefield. We did a great job avoiding UB by following those coding standards. There is some verbosity that can be avoided but hey these are details. Personally, I have moved to Rust since then and I am not looking back.

andrewgazelka 3y ago
Is there something like BMQ but in Rust? Seems interesting you mentioned it.

stolsvik 3y ago

I find it curious that lots of people always drag in Kafka as comparison to message brokers. Their own architecture (log of messages, vs. delivery of messages), but in particular the system architecture they lead to when used (event sourcing, vs. "react to events and commands"), are extremely dissimilar. Kafka has its uses, in particular for massive influx of events, e.g. in a large IoT system - I'd say the perfect example would be continuous measurements of tens of thousands of gauges and sensors on an oil rig or any other large production system, or e.g. the energy meters in every home. But I would personally never use such a system as the inter-service communication layer for a multi-service architecture. It is WAY to heavy coupling. Event sourcing looks fantastic on the surface, but is a disaster for a decades-living system with tons of developers. REST/gRPC is better, but async messaging really rocks.

marcelius 3y ago
How is Kafka causing heavy coupling? Serious question.
- stolsvik 3y ago
  If you use Kafka in an event sourced fashion, whereby each service emits events that it puts on a bunch of Kafka topics, and each service that needs that information consumes those events (by reading and applying those events to its local understanding of the world), I argue that you have effectively made a massive distributed and shared database - shared between all the services. This goes smack in the face of the idea of "one service, one database" 1), where there is a distinct boundary where each service owns their own data. How a given service stores it data is of no concerns to any other service - the communication between them is done using e.g. REST or messages, with a clear contract. Event sourcing / Kafka architectures are the exact opposite of a clear contract. You are exposing the absolute, deep-down innards of the storage solution of a service, by putting its microscopic events down for all to see, and all to consume. You may do aggregations, and emitting more coarse-grained "events" or state changes, thus kinda also exposing "views" of these inner tables, and maybe use a naming scheme to sort of which are "public" and which are not. In the beginning, I really did find the concept of event sourcing to be extremely intriguing: Both the "you can get back to any point in history by replaying up-to", and "forget the databases, just emit your state changes as events" (I truly "hate" databases, as they are the embodiment of state, which is the single one thing that makes my field hard!), the ability to throw up a new projection for anything you'd need (a new "internal view") of the state, and that unit testing could be really smooth. I upon deeper delving into the concepts found that the totality of such a system must quite quickly become extremely heavy: The amount of events, thus needing snapshots. Evolution of events, where you might have no idea of who consumes them (that is, the "shared database" problem). The necessary understanding, throwing a half-century or more of relational databases under the bus. The performance tweaking if things start to lag. Etc etc etc. It would become a distributed system in the worst way a distributed system could be distributed: All state laid out in minute details, little abstractions, and a massive diverse set of different implementations in the different services to get back to a relational view of the data. And this is even before the system gets a decade old, with lots of different developers coming and going, all of them needing to get up to speed, and the total system needing extremely good and hard steering to not devolve into absolute utter chaos. That Kafka can be employed and viewed as a database has been argued hard by Confluent. Here's their former DevEx leader Tim Berglund explaining how databases are like onions, and that in the base of every database there is a commit log. And that this log is equivalent to a set of Kafka topics. 2) So why not just implement all the other layers of the database in application logic? Confluent even have a lot of libraries and whatnots to enable such architectures. 1) "Microservice Architectures", patterns: https://microservices.io/patterns/data/database-per-service.html https://microservices.io/patterns/data/database-per-service.... 2) JavaZone Talk from 2018 by Tim@Confluent: https://2018.javazone.no/program/3a9644e6-15b5-4c66-a28c-c35dc4b85c87 https://2018.javazone.no/program/3a9644e6-15b5-4c66-a28c-c35...

stolsvik 3y ago

I'm the developer of the Message-Oriented Async RPC library Mats3: https://mats3.io/ https://mats3.io/ Its current sole implementation is based on Java Messaging Service JMS API, and it is used in production for a rather large UCITS unit holder registry on Apace ActiveMQ, and all tests runs fine on Apache Artemis MQ. Every time a new message broker comes along, I sit up in the chair and wonder a) about their performance (!), and b) whether they have a JMS client implementation, and c) whether Mats3 works with that! When I tested RabbitMQ's JMS client library, I sadly found that there was rather many differences - things I thought was screamingly obvious, was not available. E.g. as basic function as redelivery: "Normal" MQs try to redeliver N times, typically with a backoff between each attempt, and then, when all N attempts fail, it puts the message on the DLQ. Rabbit instead tight-loops the delivery attempts until either the message goes through, or the sun burns out. To be able to use Rabbit, I would have to use the native API, and implement redelivery and DLQing myself, client side. Also, transactions. I now wonder whether I should make a lower-level abstraction, so that the JMS implementation is converted to a "base" impl, and then the JMS, Rabbit, Bloomberg, NATS, ZeroMQ, Aeron, etc implementations was extensions, or "plugins", or "drivers", to that.

the-alchemist 3y ago
Sounds great, and you have lots of nice documentation on the page, but could you provide a TLDR? There's a lot of competition in this area: GRPC, Cap'n'proto (was posted on HN a day or two ago), NATS, etc. I'm also having trouble figuring out if Mats3 is a library (with a JMS API) over a variety of messaging systems (WebSockets, NATS, etc.)? P.S. Some diagrams like https://bloomberg.github.io/blazingmq/ https://bloomberg.github.io/blazingmq/ would be very helpful, especially at https://mats3.io/background/what-is-mats/ https://mats3.io/background/what-is-mats/. If a picture's worth a thousand words, and an animation must be worth at least 10k words. :)
- stolsvik 3y ago
  I am truly finding it hard to explain it. I have tried along a dozen angles. I actually think it is a completely new concept - and that might be the problem. But there is actual value here, because it "merges" the amazingness of using messaging, with the simplicity of "linear thought" that synchronous, blocking code gives (i.e. ISC using e.g. REST or gRPC). #) There is an illustration on the front-page: https://mats3.io/ https://mats3.io/ This page tries to directly explain the idea - but I guess it assumes that the reader is already extremely familiar with message queuing? https://mats3.io/docs/message-oriented-rpc/ https://mats3.io/docs/message-oriented-rpc/ Here's a set of small answers, from different angles, to "What is Mats?": https://github.com/centiservice/mats3/blob/main/README.md#what-is-mats https://github.com/centiservice/mats3/blob/main/README.md#wh... Here's a way to code up Mats3 endpoints using JBang and a small toolkit which makes it extremely simple to explore the ideas - the article should also be skimmable even without a command line available: https://mats3.io/explore/jbang-mats/ https://mats3.io/explore/jbang-mats/ If you read these and then get it, I would be extremely happy if you gave me a sentence or paragraph that would have led you to understanding faster! The concept of messaging with queues and topics are essential to Mats3 - but the use of JMS is really just a transport. I could really have used anything, incl. any MQ over any protocol, or ZeroMQ, or plain TCP - or just a shared table in a database (but would then have had to implement the queue and topic semantics). As a developer, you code Mats3 Endpoints by using the Mats3 API, and you initiate Mats3 Flows using the API. You need an implementation of the MatsFactory to do this, and the sole existing is the JmsMatsFactory - which works on top of ActiveMQ and Artemis's JMS clients. Wrt. WebSockets, that is a transport typically between a server, and a end-user client, e.g. an iOS App. Actually, there's also a "sister project", MatsSockets, that bring the utter async-ness of Mats3 all the way out to the client, e.g. a webpage or an app. https://matssocket.io/ https://matssocket.io/ NATS is, AFAIU, just a message broker, with some ability to orchestrate. Fundamentally, I do not like this concept of orchestration as an external service - this is one of the founding ideas of Mats3: Do the orchestration within each service, as you would do if you employed REST as the ISC mechanism. I do however assume that one could make a Mats3 implementation on top of NATS client libs. #) Java's Project Loom is extremely interesting to me, as its argument for using threads as the basis of concurrency instead of asynchronous styles of programming, is exactly the same rationale for which I made Mats3: It is much simpler for the brain to grok a linear, sequential, "straight-down" code, than lots of pieces of code that are strung together in a callback-hell. Async/await is a way to make such callback-hell more readable, "linearaize it" (but you still have the "colored methods" problem which Loom just obliterates in a perfect explosion). One could argue that this is what I have achieved with Mats3 when using messaging.