3 ms·

(I'm the PM for this at Cloudflare & the blog author) We looked at a number of existing options — trying to strike a balance between languages we run at scale

(I'm the PM for this at Cloudflare & the blog author) We looked at a number of existing options — trying to strike a balance between languages we run at scale (tooling, ops, security), memory safety, and long-term maintainability as a growing company. We ended up writing an MQTT broker from scratch in Rust, with our own Durable Objects[1] for consistency and co-ordination across our hundreds of edge locations (and tens of thousands of machines). The MQTT spec itself[2] is very approachable, which made this undertaking easier still. One of the increasingly cool parts of Cloudflare is that we use more and more of what our customers use: our Broker service sits behind Spectrum[3], and we use Workers + DO for all of the metadata/session handling. The next set of new products will invariably start to build on Pub/Sub itself, too :-) [1]: https://developers.cloudflare.com/workers/learning/using-durable-objects/ https://developers.cloudflare.com/workers/learning/using-dur... [2]: https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html [3]: https://developers.cloudflare.com/spectrum/ https://developers.cloudflare.com/spectrum/

fasteo 4y ago
Thanks, make sense. Actually, I asked because it surprised me that D1 - sqlite - was not written the same way. [1] https://blog.cloudflare.com/introducing-d1/ https://blog.cloudflare.com/introducing-d1/
- elithrar 4y ago
  D1 _does_ use Durable Objects under the hood! Like Pub/Sub, it's ultimately an implementation detail, albeit probably interesting to this audience.
  - fasteo 4y ago
    >>> And, since we're building on the redundant storage of Durable Objects, your database can physically move locations as needed, resulting in self-healing from even the most catastrophic problems in seconds. True. I missed that part ! Thanks.
maxpert 4y ago
Do you plan to opensource the implementation some time?
- elithrar 4y ago
  No concrete plans at this time, especially since there are many competing implementations. If we open-sourced we’d need to do it right (proper governance, triage, support, etc) vs just putting into a public repo and calling it a day.
fisian 4y ago
Some time ago I was at a talk about verifying different MQTT brokers so they fulfill the spec. They presented several inconsistencies they found (in OSS brokers) and submitted fixes for. I looked up if there's some info online and found that paper (unfortunately paywalled): https://graz.pure.elsevier.com/en/publications/learning-based-fuzzing-of-iot-message-brokers https://graz.pure.elsevier.com/en/publications/learning-base...
- elithrar 4y ago
  The F-Secure folks also have https://github.com/F-Secure/mqtt_fuzz https://github.com/F-Secure/mqtt_fuzz - although it doesn't support authenticated brokers as of yet.
kitd 4y ago
The MQTT spec itself[2] is very approachable, which made this undertaking easier still. Indeed. I've found writing an MQTT client is a good tutorial project when learning a new language.

fasteo 4y ago

Thanks, make sense. Actually, I asked because it surprised me that D1 - sqlite - was not written the same way. [1] https://blog.cloudflare.com/introducing-d1/ https://blog.cloudflare.com/introducing-d1/

elithrar 4y ago
D1 _does_ use Durable Objects under the hood! Like Pub/Sub, it's ultimately an implementation detail, albeit probably interesting to this audience.
- fasteo 4y ago
  >>> And, since we're building on the redundant storage of Durable Objects, your database can physically move locations as needed, resulting in self-healing from even the most catastrophic problems in seconds. True. I missed that part ! Thanks.

maxpert 4y ago

Do you plan to opensource the implementation some time?

elithrar 4y ago
No concrete plans at this time, especially since there are many competing implementations. If we open-sourced we’d need to do it right (proper governance, triage, support, etc) vs just putting into a public repo and calling it a day.

fisian 4y ago

Some time ago I was at a talk about verifying different MQTT brokers so they fulfill the spec. They presented several inconsistencies they found (in OSS brokers) and submitted fixes for. I looked up if there's some info online and found that paper (unfortunately paywalled): https://graz.pure.elsevier.com/en/publications/learning-based-fuzzing-of-iot-message-brokers https://graz.pure.elsevier.com/en/publications/learning-base...

elithrar 4y ago
The F-Secure folks also have https://github.com/F-Secure/mqtt_fuzz https://github.com/F-Secure/mqtt_fuzz - although it doesn't support authenticated brokers as of yet.

kitd 4y ago

The MQTT spec itself[2] is very approachable, which made this undertaking easier still. Indeed. I've found writing an MQTT client is a good tutorial project when learning a new language.