17 ms·

Mosquitto: An open-source MQTT broker

271 points by ducktective 5y ago

tombert 5y ago
I've never used Mosquitto, but I have done a fair amount with Verne.mq [1], and I have to say that MQTT is downright pleasant to use in a lot of cases. I've not done a ton with it in "real world" situations, but I have used it for multiple hackathons, and I'm always impressed how little of a headache it is to build a decent "live" application with almost no effort. If your frontend web project calls for any kind of messaging, I definitely recommend looking into trying MQTT before you jump straight into WebSockets. There's a good chance MQTT does what you need, scales better, can communicate over WebSockets, and will make your life easier . [1] https://vernemq.com/ https://vernemq.com/
- PostThisTooFast 5y ago
  Yeah, it's pretty cool. I am tasked with developing a new API and communication method for a bunch of networked devices that run embedded Linux. Initially I focused on MQTT, but have since decided to define an API in OpenAPI and implement an HTTP/REST-style interface because that's well understood and easily supported by third parties, whose APIs we might want to integrate with or provide a translation layer for. But the devices need to alert a controller application to status changes... which so far I supposed they would do by POSTing REST-style messages back to that application. Do people combine REST & MQTT?
  - eska 5y ago
    The REST paper explicitly mentions that REST and HTTP only try to solve the common cases, and solve those well. If something is not covered well then you’re explicitly advised to leave it to another protocol. You could for example have a GET response that (due to HATEOAS) suggests a websocket url including a topic to subscribe to.
  - serial_dev 5y ago
    > Do people combine REST & MQTT? I'm not a REST+MQTT expert, but some people do combine them, why not? One of the project I was working on (IoT, smart home), the mobile app received the current status from the REST API, then subscribed to changes via MQTT. Having MQTT was great for live updates on the mobile app, and for communicating with the IoT devices. HTTP was great for integrations with Google Home, Alexa, and we could test it easily as there are many backend frameworks focusing on CRUD REST HTTP backend apps. Of course, if you actually have a problem you want to solve, be sure the to do your own research, there are more than one way to skin a cat, and there are so many services and platforms that could be "good enough" for your use case.
    - PostThisTooFast 5y ago
      Thanks! This is similar to the scenario I'm talking about. I'm mostly presenting a remote control to the user and sending commands to the device. But I also need to get status from the device, and polling for the state of every setting is a crappy way to go.
  - kitd 5y ago
    One thing to note is that MQTT is session-oriented, so you have to send periodic pings to the server to keep your session active. Which is obviously a deviation from stateless REST.
  - zoom6628 5y ago
    Generally people choose MQTT because it is lightweight and very well tested. I personally use it because its light one source, scalable, and very flexible. Have also chosen to use CoAP rather than REST for ad hoc data requests because for constrained devices it is much lighter on bandwidth, simpler to use than typical REST over TCP+HTTP with which most people are familiar. In IOT its hard to go past MQTT for the usual signals paired with CoAP for ad hoc. When I want to reduce dependencies or just being lazy even CoAP gets replaced with building a request/respond mechanism using a MQTT queue so a hub pubs a message to the devices command queue and device reads that queue and responds to it. How often device checks the command queue depends on whether or not device can run command queue monitoring asynchronously or must do a scheduled round-robin of the queues. The Eclipse Paho libs have been great for me using across different OS and bare metal, Arduino, FreeRTOS.
    - PostThisTooFast 5y ago
      Thanks. My application doesn't require ad-hoc queries; it's all about setting parameters on a device and getting status back.
- jvanderbot 5y ago
  I have used ZMQ for this with great success. The one-line python calls to send data between nodes are really nice.
  - alias_neo 5y ago
    ZeroMQ is fantastic. I've been using at work for a few years now, but I generally enjoy the tech anyway, their documentation[0] and "The Guide"[1] is also really nice, with examples for all of the different language bindings. From C or Golang (or a bunch of other language bindings) you can set up some nice pub/sub or req/res with fan in/fan out and all sorts of cool routing with low-level, high speed sockets with the added power that ZMQ provides, such as the queueing, high-watermarks etc. You can also enable encryption easily with CurveZMQ which is a refreshing change from TLS. [0]https://zeromq.org/get-started/ https://zeromq.org/get-started/ [1]https://zguide.zeromq.org/ https://zguide.zeromq.org/
    - tombert 5y ago
      I almost feel like the guide should be mandatory reading for anyone learning network programming. It gives extremely digestible summaries of most of the network patterns, when and why you'd use them, and then how they're done in ZeroMQ. Even if you end the guide thinking that ZeroMQ is stupid (which you probably wouldn't), you'd still probably benefit from having read the guide.
  - tombert 5y ago
    I love ZeroMQ, but I didn't think that had any first-class support for communicating in the browser (could be wrong). I agree it's super nice though, even with a single process.
- RedShift1 5y ago
  The VerneMQ bugtracker scares me. Especially this one: https://github.com/vernemq/vernemq/issues/1663 https://github.com/vernemq/vernemq/issues/1663. I'm running one instance but I'm on the lookout for an alternative that can more reliably save messages if a subscription client goes down.
  - tombert 5y ago
    I wish I could say that I chose Verne.mq because I weighed all the options and it was the best for my project but...I really chose it because it was written in Erlang and I like Erlang. I won't pretend to have done an audit of the best servers in terms of bugs or features.
- mosselman 5y ago
  > I definitely recommend looking into trying MQTT before you jump straight into WebSockets. What is a nice way to use mqtt in the front-end instead of websockets? What can you recommend?
  - oneweekwonder 5y ago
    Eclipse Paho JavaScript Client[0] for the front-end and personally I prefer RabbitMQ Web MQTT Plugin[1] for the back-end[1]. Take note Paho MQTT Client is available for several programming languages(Py,Rust,Go,C,C#,etc)[2]. [0]: https://www.eclipse.org/paho/index.php?page=clients/js/index.php https://www.eclipse.org/paho/index.php?page=clients/js/index... [1]: https://www.rabbitmq.com/web-mqtt.html https://www.rabbitmq.com/web-mqtt.html [2]: https://www.eclipse.org/paho/index.php?page=downloads.php https://www.eclipse.org/paho/index.php?page=downloads.php
    - tombert 5y ago
      Beat me to it! I was about to suggest the Eclipse Paho library. One of the very few things in the developer world that has "just worked" for me the first time I tried it.
- deleted 5y ago
  [deleted]
- SirMonkey 5y ago
  I learned about them on my previous job. It's rather easy to cluster and the Web-Hooks are super easy to get started with. I don't know Erlang myself, but looking at the code and comparing it to some of the Java solutions, it seemed well structured and easier to understand/modify if I wanted to. The only "Issue" I have with it right now, is the "custom OpenSource license" they have.
IceWreck 5y ago
I'm a big fan of NATS. It a small binary which can be scaled as much as you want and it replaces Kalfka, lightweight MQTT brokers, etc and has clients for most languages.
- ducktective 5y ago
  https://nats.io/ https://nats.io/ https://github.com/nats-io/nats-server https://github.com/nats-io/nats-server Is it suitable for embedded applications? (very low resources, intermittent network availability, etc)
  - AkshitGarg 5y ago
    The NATS protocol [0] looks pretty simple, but is text based unlike MQTT which uses a size optimized binary protocol. [0]: https://docs.nats.io/nats-protocol/nats-protocol https://docs.nats.io/nats-protocol/nats-protocol
  - DenseComet 5y ago
    NATS actually also has support for MQTT[0]. If you already have a deployment for queues, microservices, etc, using the NATS mqtt endpoint seems like a very good option. [0] https://docs.nats.io/nats-server/configuration/mqtt https://docs.nats.io/nats-server/configuration/mqtt
pletnes 5y ago
I take it mqtt was developed for embedded systems, microcontrollers etc to be lightweight. Can the broker also run in such and environment, or is that just the clients?
- zorr 5y ago
  Mosquitto is C/C++ without any crazy dependencies so you should be able to do that. I don't immediately see why you would do this though? Any specific use cases? I haven't done this myself yet but I've seen a couple of production IoT setups with a local MQTT broker running on a RaspberryPi configured to relay all messages to a central broker running in the cloud.
  - jvanderbot 5y ago
    Pi is a far cry from "Microcontrollers". MQTT requires a TCP stack, which limits to the higher end chips. But, for embedded linux, you're probably golden.
    - joezydeco 5y ago
      MQTT is available via LWIP on STM32 parts that don't have an MMU. I've tried it on the STM32F746 (Cortex-M7) and it works well, although there are other issues with networking on CubeMX that are open.
- detaro 5y ago
  If you don't need it to scale crazily, yes. E.g. microcontrollers in nodes and then a local gateway as an embedded Linux system would not surprise me to see.
- joezydeco 5y ago
  I'm running Mosquitto as the central messaging broker on an embedded Linux product right now. It's super lightweight. Running the broker on the target has been a really great thing during development. I can run validation systems on other machines and inject messages over the wire, or I can actually run the UI on my dev machine and let the backend respond from a live target. I also have a development team in Asia that can't get to the newest hardware, so I fire up a system here and open a port they can access. They love it. That said, the bandwidth in MQTT is lightweight but you still need a TCP/IP stack on your system. That's typically more substantial of a footprint than Mosquitto will be at the end of the day.
deknos 5y ago
is mosquitto clusterable nowadays? on its own or via k8s? for: * fault tolerance * fail over * vertical scalability? does anyone know this? if not is there a another opensource mqtt broker which does that?
- mlhales 5y ago
  Mosquitto is not, we use RabbitMQ which runs on Erlang/OTP and has mqtt plug-ins. Mosquitto is single threaded. It's fast for sure but doesn't scale out. They can interoperate though as Mosquitto has configurable bridging.
- eska 5y ago
  There are broker implementations that support clustering. Emqx comes to mind.
- serial_dev 5y ago
  VerneMQ [0] could be interesting for you, they claim that: > VerneMQ is a high-performance, distributed MQTT broker. It scales horizontally and vertically on commodity hardware to support a high number of concurrent publishers and consumers while maintaining low latency and fault tolerance. I'm not sure about their licensing at the moment [1], to me it's a bit confusing. [0] https://vernemq.com/ https://vernemq.com/ [1] https://vernemq.com/blog/2019/11/26/vernemq-end-user-license-agreement.html https://vernemq.com/blog/2019/11/26/vernemq-end-user-license...
- jonquark 5y ago
  There are lots of brokers and Eclipse Amlen is coming soon: https://www.eclipse.org/amlen/ https://www.eclipse.org/amlen/ (disclosure: I work on Amlen)
- ralight 5y ago
  I work for Cedalo AG developing Mosquitto. Cedalo will be offering a version of Mosquitto with failover support as a paid for feature soon.
- oneweekwonder 5y ago
  Recently wanted to also create a cluster of mqtt brokers and found this medium tutorial stringing Docker Swarm, RabbitMQ, Consul and HAProxy together very helpful[0]. I was not happy with the static ip's for haproxy and opted to register the mqtt/tls services in consul and use the consul dns resolver with hapraxy. [0]: https://medium.com/hepsiburadatech/implementing-highly-available-rabbitmq-cluster-on-docker-swarm-using-consul-based-discovery-45c4e7919634 https://medium.com/hepsiburadatech/implementing-highly-avail...
zerkten 5y ago
For someone that has worked with RabbitMQ, is there a guide to the differences and appropriate usage of MQTT brokers with some real use cases explained?
- oneweekwonder 5y ago
  You are aware RabbitMQ support MQTT[0] and MQTT over WebSockets[1]? As for comparing it to RabbitMQ you should compare the AMQP protocol vs MQTT protocol, it seems MQTT with its lightweight design "won" IoT market over and should be rather compared against protocols like CoAP. As for guides I have always found resources rather scarce and mostly learned by adding features to mqtt libraries or writing my own prototype games using it. [0]: https://www.rabbitmq.com/mqtt.html https://www.rabbitmq.com/mqtt.html [1]: https://www.rabbitmq.com/web-mqtt.html https://www.rabbitmq.com/web-mqtt.html
  - growse 5y ago
    Unfortunately, Rabbit's implementation of MQTT has some pretty serious deficiencies which you may find ends up being a deal-braker (e.g. https://github.com/rabbitmq/rabbitmq-server/issues/2556 https://github.com/rabbitmq/rabbitmq-server/issues/2556).
EastOfTruth 5y ago
I used Mosquitto years ago for a few years and never had any issues with it...
zwieback 5y ago
I like MQTT and we use Mosquitto in our product but I always wonder if MQTT pushes TCP into areas where UDP would be a better fit.
- mlaretallack 5y ago
  That is where MQTT-SN comes in. I have used it over NB-IOT and then a gateway to full MQTT over mTLS to AWS IoT.
- ralight 5y ago
  Like your other reply said, MQTT-SN can use UDP so may be a better fit in some situations. It's a shame that MQTT-SN hasn't seen wider use, but there isn't that much support at the moment. The protocol is being standardised at the moment, and there has been an MQTT to MQTT-SN gateway as part of the Eclipse Paho project for a while now. The Mosquitto project has the aim of adding support for MQTT-SN - it's just not as high a priority as other things at the moment unfortunately though. If it's something that's important to you, please get in touch. Roger
clumsysmurf 5y ago
Anyone have experience comparing MQTT with NATS, they seem to cover similar scenarios.
- JesseObrien 5y ago
  Would be looking for this as well. I love using NATS, would like to see a comparison of both.
eska 5y ago
Unfortunately the websockets implementation only allows IPv6, and in my tests from a js app sometimes took seconds to start if it worked at all. I’m currently looking for an alternative broker due to those issues.
- oauea 5y ago
  > Unfortunately the websockets implementation only allows IPv6 Huh? Pretty sure that's not true.
  - eska 5y ago
    >socket_domain [ ipv4 | ipv6 ] >By default, a listener will attempt to listen on all supported IP protocol versions. >If you do not have an IPv4 or IPv6 interface you may wish to disable support for either of those protocol versions. In particular, note that due to the limitations of the websockets library, it will only ever attempt to open IPv6 sockets if IPv6 support is compiled in, and so will fail if IPv6 is not available. From the manual. Huh, seems like I might’ve misunderstood this paragraph. But I was only ever able to access the broker if the browser was using an ipv6 address. I’ll have to test again I suppose. Still, the performance and reliability was not good.
    - oauea 5y ago
      Right, that just says if you don't have IPv6 support compiled in, you can't use IPv6. IPv4 is always available, and has worked wonderfully well in my experience.
      - eska 5y ago
        Thanks for pointing that out. I’ll have to try again. Maybe ipv4 got disabled in the configuration somehow by accident.
- ralight 5y ago
  As others have pointed out, that's not the case. Something you do have to be careful of is the compile options for libwebsockets and what version you're using. Recent-ish version of libwebsockets in particular disable by default some features Mosquitto relies on, so some installations may suffer. Version 2.1 of Mosquitto will no longer need libwebsockets (although it can still be selected at compile time) and so websockets support will be less dependant on how other libraries are compiled.
rhodin 5y ago
We run a hosted version starting at $5/month [0]. We used to have a free version but it was too popular. [0] https://www.cloudmqtt.com/plans.html https://www.cloudmqtt.com/plans.html
- jshier 5y ago
  The free version was a great resource while developing MQTT clients a few years ago, sorry it was too expensive.
  - rhodin 5y ago
    If you are a library developer/maintainer we'd be happy to sponsor you with free subscriptions, contact us at support@cloudmqtt.com.
- jayzym 5y ago
  We use this for work and it's a pretty slick product. Please add support for MQTT 5.0, or shared subscriptions!
- pmx 5y ago
  We use cloudmqtt at work and we've been really impressed by the price and quality of the product. You made it not worth hosting our own instance!
- jasonvorhe 5y ago
  In case someone's looking for a free MQTT service with the only constraints being client connection limits and bandwidth, check out HiveMQ Cloud: https://hivemq.cloud https://hivemq.cloud
Kuinox 5y ago
I'm developing a mqtt client/broker and the protocol is all but simple to implement. I didn't found any implementation, mosquitto included, that was really lossless.
- RedShift1 5y ago
  The protocol is simple (I've written my own implementation in php), what do you mean with lossless?
  - Kuinox 5y ago
    That when you use QoS 2 "ExactlyOnce" (or 1), you will never loose a message.
- luma 5y ago
  Is that true if you set QoS to a value other than 0 in your application? https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels/ https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-qual...
  - Kuinox 5y ago
    No, even when you set the QoS to 2, "ExactlyOnce", you may still hit edge cases where you loose data, without knowing it, especially on enterprise networks, or if there is an high throughtput.
- xnyan 5y ago
  I've worked quite a bit with MQTT - can you tell us a little more about what you mean by "really lossless"?
  - Kuinox 5y ago
    To never loose a packet on QoS 1/2.
- nicolaslem 5y ago
nicolaslem 5y ago
MQTT is becoming the defacto interface for home automation. As an example, Home Assistant used to have plugins for zwave and zigbee but they are now replaced by an MQTT broker that sits between Home Assistant and the program driving the wireless dongle. Since I'm already running Mosquitto to interface with zwave devices, I decided to also use it to monitor my home lab. Servers push sensor statuses into Mosquitto and Home Assistant is used for the UI and alerting. That was a fun week-end project.
- oynqr 5y ago
  ZHA is still the preferred way for Zigbee with Home Assistant, isn't it?
  - EngCanMan 5y ago
    Many people use https://www.zigbee2mqtt.io/ https://www.zigbee2mqtt.io/ now too, it seems people are moving in this direction
  - dusing 5y ago
    ZHA is great and fast, I prefer it instead of zigbee2mgtt. I have 100+ zigbee devices and 80 z-wave.
  - nathan_f77 5y ago
    I started with ZHA but had problems with some devices (e.g. a 4-way switch panel that would only show as a single button.) I've had a much better experience with zigbee2mqtt, and it seems a bit more powerful
  - specto 5y ago
    Saw this too recently - https://www.cnx-software.com/2021/09/27/zb-gw03-esp32-based-ethernet-zigbee-gateway-now-working-with-tasmota-firmware/ https://www.cnx-software.com/2021/09/27/zb-gw03-esp32-based-..., quite nice
  - jhot 5y ago
    I've been happy with the ZHA implementation and haven't tried z2mqtt. I am much happier with zwavejs than the old implementation, though. I do run MQTT for a few wifi devices and communication between node red and custom client code on my laptop/desktop.
SavantIdiot 5y ago
Anyone have experience scaling with more than 256 devices? I use MQTTS at work and for my own projects. It is a very thin protocol (low code footprint), especially combined with Wi-Fi where those stacks typically eat up most of your MCU's flash if you're using a single-chip solution with TLS. I've used it on a max of 256 devices on a single BSSID, but I would be curious to know how it scales. EDIT: Specifically, when all devices are subbed to a noisy channel. In some cases we had to stagger pub with random jitter to prevent both Wi-Fi and edge processing issues.
- eska 5y ago
  I’ve only seen papers from the automobile industry where they used MQTT but fed it into Kafka to scale.
- IshKebab 5y ago
  All Dyson smart products use MQTT. As I recall it works fine from a performance point of view (it's a very lightweight protocol), but personally I thought it was a pretty flawed choice because the problem of choosing/discovering paths is just completely ignored, and it shares the same problem as all event bus style systems - events can get lost, it's very difficult to debug and learn, etc.
  - SavantIdiot 5y ago
    > events can get lost, Well, yes, but QoS=1 or 2 with a reasonably large queue I haven't seen loss issues. What kind of loss problems are you running into (that I should be worried about)?
    - IshKebab 5y ago
      Yeah I don't mean due to the protocol, I mean due to application bugs where something adds a message to some queue and the thing that is meant to process it doesn't for some reason, and then nothing will ever realise. Compared to callbacks / RPC for example, where you have a function call the either has to succeed or it has to fail. It's not a deal breaker but I don't like it.
mleo 5y ago
MQTT/Mosquitto relays is a great solution for passing data across network boundaries when inbound is limited. I use it with Webhooks deployed to Internet VM that pass the data to broker installed on VM. Another broker in home network connects to Internet broker and relays messages to local clients. I don’t have to open any ports into my home network for webhooks to work with other services.
jvanderbot 5y ago
I've recently transitioned my home "IOT" network (and servers) to a Wireguard-only network. It's wonderful. All the routing accessibility through firewalls or with dynamic IPs just disappeared. All services are exposed only on the wireguard interface, meaning security and connectivity are solved (to the 80/20 level) with just Wireguard. That kind of removed the need for a static broker entirely since now every node has a static IP on my wireguard network and are accessible no matter what network they are actually on. Pinging my mobile phone from an air quality sensor was an eye-opening experience.
- kobalsky 5y ago
  what kind of IOT devices are we talking about? linux embedded or is wireguard available for something like nodemcu?
  - jvanderbot 5y ago
    Linux embedded. (Pis mostly)
- 3np 5y ago
  Did you ever notice any issues with significant increased softirqs? I started going down a similar path but a node that is acting as a gateway/bridge for UDP traffic routing to an outbound wireguard tunnel is now spending a constant ~20% CPU time on NET_RX softirqs with 10Mbps and 5~30Kpps. Almost no NET_TX. It's specifically the combination of UDP-over-wg that seems to cause this. It's otherwise handling 1Gbps and way more packets just fine. (I'm aware MQTT is TCP but hey, worth a shot)
- jve 5y ago
  Hey, tell us more, please :) I'v hoped to find in your profile maybe more info/blog, but instead: > Group supervisor at NASA Jet Propulsion Laboratory working in autonomous vehicles and sensing for space exploration. Oh well, a little out of scope and I see you blog about other things. Anyway - you mean I don't need static IP to form a network of distant IoT devices? This would be some really useful property.
travisgriggs 5y ago
I'll just pile on my own accolades. We use MQTT to connect handhelds with in the field ag irrigation controllers. There's probably about 10,000 valves on farms and other ag projects throughout, that are ultimately scheduled and monitored through MQTT. We've even been able to build a full request/event protocol on top of it. The protocol is simple enough, that when I wasn't happy with not many full solutions existing in iOS, I just rolled our own in a day or two. Currently adding an Elixir REST bridge to the whole thing. It's been pretty cool all around.
- ralight 5y ago
  That sounds really nice, I love interesting projects in areas outside of my experience. If you're using Mosquitto I'd love to hear more about it - my contact details are at the bottom of https://github.com/eclipse/mosquitto/ https://github.com/eclipse/mosquitto/ :)
bsmth 5y ago
I really like Mosquitto, I'd recommend it for quickly getting started hacking together IoT projects on the hobby / side project scale. One of our community (QuestDB) contributors put together a docker compose example that I thought was pretty nice for quick prototyping, could be convenient for someone here: https://github.com/shantanoo-desai/questitto https://github.com/shantanoo-desai/questitto
monkeydust 5y ago
Use Mosquitto on Windows server at home to transfer data from my energy smart meter to home assistant. Pretty rock solid and easy to diagnose issues via mqtt tools.
ajfally 5y ago
I've been wanting a GCM/FCM alternative, could mqtt be used for notifications like that? I understand its usually used for IOT devices and stuff so I assume its suitable for not using loads of power
- malermeister 5y ago
  Thats what Facebook Messenger uses. Downside to this is that only FCM can work around android's power saving mechanisms and deliver instantly... unless you're Facebook and able to convince vendors to patch the firmware for your servers too ;)
  - holoduke 5y ago
    Yep. Another real concrete example of an unwanted monopoly. We have issues with fcm, we are sending about 10m per day for millions of users. About 1 to 3% never arrives. Ends up in the void. Google doesn't want to help. We are not Facebook. But no alternative. A pity
    - malermeister 5y ago
      97% is actually a really good delivery rate. From my experience working on several push-based products, ~80% is the industry standard.
nathan_f77 5y ago
I've learned a lot about MQTT while setting up Home Assistant, and Mosquitto is a pretty important part of my Home Assistant installation. I use it to control all of my Zigbee devices via "zigbee2mqtt" (buttons, switch modules, etc.), and also a smart kettle running Tasmota [1]. I also added a little patch to Tasmota [2] so that my kettle responds with `418 I'm a teapot` [3] [1] https://tasmota.github.io/docs https://tasmota.github.io/docs [2] https://github.com/ndbroadbent/Tasmota/commit/bbcf57faffbf5b7fab5c54f395b7b720f8df50b6 https://github.com/ndbroadbent/Tasmota/commit/bbcf57faffbf5b... [3] https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/418 https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/418
- seidoger 5y ago
  Prime use-case for HTTP 418!
  - SV_BubbleTime 5y ago
    People always want that for teapots. I have a device that uses it as an indication it’s internal data shouldn’t be trusted (some unknown error). I have it respond with 418 as “I taste colors and the government spies on you with the ground in your house wiring”. I like it for that more than actual tea pots which is far too on the nose.
    - Something1234 5y ago
      What kind of device might have internal data that cannot be trusted?
  - deleted 5y ago
    [deleted]
- matttrotter 5y ago
  Cool stuff and inspiring. Have a website or GitHub about your home automation projects? I checked out your blog, and the raspberry pi microwave was pretty sweet.
bborud 5y ago
This project is exceptionally useful not only in itself but as a tool you can use during development. Software that can be controlled entirely from the command line without configuration files (unless you choose to) are very convenient.
boilerupnc 5y ago
The node-red [0] project is a great complement to MQTT use-cases. [1] "Node-RED is a programming tool for wiring together hardware devices, APIs and online services in new and interesting ways. It provides a browser-based editor that makes it easy to wire together flows using the wide range of nodes in the palette that can be deployed to its runtime in a single-click." [0]: https://nodered.org/ https://nodered.org/ [1]: https://cookbook.nodered.org/mqtt/connect-to-broker https://cookbook.nodered.org/mqtt/connect-to-broker
- deepspace 5y ago
  Node-RED is one of those polarizing apps. People either love it or hate it with a passion. I am in the latter camp. I mean, it does what it says on the tin, but I could never understand how laboriously "wiring" up a bunch of boxes and filling in parameters is any easier than whipping up a few lines of Python.
  - sleavey 5y ago
    I'm in your camp too, but would it be fair to say that Node-RED is not designed for us?
  - EmpirePhoenix 5y ago
    Well, if you are not a code based developer, eg. you are used to program industrial control systems, they are often developed the same way, as with certified/proofed tools, you can guarantee that the code has no uncaught error paths or simple syntax errors. It also has a lot appeal to people who do not develop on a daily basis, eg. I'm pretty confident I could teach my dad to use nodered vs writing python in a day. I myself just use it (after years of being on the second side) because I noticed, that it was somehow way faster to get stuff working and keep it maintained
  - slightwinder 5y ago
    Isn't it because the boxes have in-place documentation and the wires visualize the workflow? Making it easy & fast for the occasional user and let them avoid the pitfalls of unmastered syntax.
yasserf 5y ago
MQTT is awesome. In deepstream.io we bridged the node MQTT protocol to deepstream which allows us to mix and match protocols. MQQT for IoT devices (and to be honest any device considering the large amount of libraries) and the deepstream JS client for node/browsers. This pretty much allowed clustering out of the box + automatically saves messages with quality assurance set in an actual database. Plus a bit more authentication and permission tools. https://deepstream.io/blog/20190909-release-deepstream-4.2/ https://deepstream.io/blog/20190909-release-deepstream-4.2/ Deepstream itself is under maintenance right now, but I find the concept of mixing protocols together (similar to what ably.io provides) to hit a sweet spot for development across multiple different environments/devices
canada_dry 5y ago
Perhaps someone here that's familiar with MQTT (and variants) can shed light on how to handle this case: an IoT device that periodically comes to life, queries the broker for the last known state of topic 'xyz', then sends a message regarding another topic 'abc' and goes back to sleep. i.e. it's not awake to receive updates to the 'xyz' topic, but needs to know the last update when it awakes.
- Kuinox 5y ago
  With mqtt you may do that with retained messages. The emitter of a message mark the message as to be retained. The server will retain it. Now when your mqtt client will subscribe tout the topic, the server will send the last retained message for this topic.
jpgvm 5y ago
If you need a highly available broker you can consider https://github.com/streamnative/mop https://github.com/streamnative/mop which adds a MQTT protocol handler to Apache Pulsar. I recently contributed auth support and TLS support also just landed. There is perhaps a few non-standards compliant things that need to be worked out but it's in fairly good shape.
noisy_boy 5y ago
I think it is time to use Kafka for home automation messages.
- adolph 5y ago
  Enterprise-class home automation https://developer.ibm.com/tutorials/kafka-in-kubernetes/ https://developer.ibm.com/tutorials/kafka-in-kubernetes/
BrandoElFollito 5y ago
I started home automation in my late 40-ies, as an amateur developer. Despite having casually developed for 25 years at that point, I found it amazing how much I learned quickly when setting up Home Assistant and pyscript (and obviously mqtt). This drove me to use a bus (mqtt for instance) for non-home automation code just because it is so useful. To friends who wonder how to help their kids get interested in coding (which is not easy at the beginning because converting farenheit to celsius gets boring after some time), I tell them to buy a few zigbee devices and have them automate stuff in pyscript. I am happy to see that some are on the way to graduate in CS :)
randomluck040 5y ago
We’re using MQTT to have different sensors publish their measurements and different user groups can subscribe to the relevant ones. I think it’s a great way to separate stuff from each other or send meaningful data away together.
ralight 5y ago
This should have been my first comment, but there you go. Hello, I'm Roger, the Mosquitto project lead. Thanks for the kind comments, I'm happy to answer questions! I work for cedalo.com developing Mosquitto and other MQTT projects. We offer Mosquitto based support, please get in touch if that's of interest to you.
- zorr 5y ago
  Thank you for your work!
- ciberado 5y ago
  I just wanted to thank you for it. Over the years it has had a big impact on all types of projects, from industrial heavy ones to raspberry-based tinkering experiments :D
consp 5y ago
We recently used mosquitto/mqtt as the broker for a timekeeping system for a student engineering event at a track and was mirrored to the web/livestream for viewing pleasure of everyone else at home (since it was during the pandemic). It worked with minimal load and without problems even when the connection was shoddy due to extremely badly configured routers by vodafone (filling up nat tables is still a thing in 2021 apperantly).
Hema0909 5y ago
I really like Bevywise MQTTRoute[1] as it is good one for home automation solutions. It allows us to create our dashboard[2] specific to our use case with pre-built widgets. The easy & flexible integration with home assistant[3] and the mobile app[4] permits us to automate our home devices. [1] https://www.bevywise.com/mqtt-broker/ https://www.bevywise.com/mqtt-broker/ [2] https://www.bevywise.com/mqtt-dashboard/ https://www.bevywise.com/mqtt-dashboard/ [3] https://www.bevywise.com/blog/mqtt-route-integration-with-home-assistant/ https://www.bevywise.com/blog/mqtt-route-integration-with-ho... [4] https://play.google.com/store/apps/details?id=com.Bevywise.mqttroute https://play.google.com/store/apps/details?id=com.Bevywise.m...

271 points by ducktective 5y ago

I've never used Mosquitto, but I have done a fair amount with Verne.mq [1], and I have to say that MQTT is downright pleasant to use in a lot of cases. I've not done a ton with it in "real world" situations, but I have used it for multiple hackathons, and I'm always impressed how little of a headache it is to build a decent "live" application with almost no effort. If your frontend web project calls for any kind of messaging, I definitely recommend looking into trying MQTT before you jump straight into WebSockets. There's a good chance MQTT does what you need, scales better, can communicate over WebSockets, and will make your life easier . [1] https://vernemq.com/ https://vernemq.com/

PostThisTooFast 5y ago
Yeah, it's pretty cool. I am tasked with developing a new API and communication method for a bunch of networked devices that run embedded Linux. Initially I focused on MQTT, but have since decided to define an API in OpenAPI and implement an HTTP/REST-style interface because that's well understood and easily supported by third parties, whose APIs we might want to integrate with or provide a translation layer for. But the devices need to alert a controller application to status changes... which so far I supposed they would do by POSTing REST-style messages back to that application. Do people combine REST & MQTT?
- eska 5y ago
  The REST paper explicitly mentions that REST and HTTP only try to solve the common cases, and solve those well. If something is not covered well then you’re explicitly advised to leave it to another protocol. You could for example have a GET response that (due to HATEOAS) suggests a websocket url including a topic to subscribe to.
- serial_dev 5y ago
  > Do people combine REST & MQTT? I'm not a REST+MQTT expert, but some people do combine them, why not? One of the project I was working on (IoT, smart home), the mobile app received the current status from the REST API, then subscribed to changes via MQTT. Having MQTT was great for live updates on the mobile app, and for communicating with the IoT devices. HTTP was great for integrations with Google Home, Alexa, and we could test it easily as there are many backend frameworks focusing on CRUD REST HTTP backend apps. Of course, if you actually have a problem you want to solve, be sure the to do your own research, there are more than one way to skin a cat, and there are so many services and platforms that could be "good enough" for your use case.
  - PostThisTooFast 5y ago
    Thanks! This is similar to the scenario I'm talking about. I'm mostly presenting a remote control to the user and sending commands to the device. But I also need to get status from the device, and polling for the state of every setting is a crappy way to go.
- kitd 5y ago
  One thing to note is that MQTT is session-oriented, so you have to send periodic pings to the server to keep your session active. Which is obviously a deviation from stateless REST.
- zoom6628 5y ago
  Generally people choose MQTT because it is lightweight and very well tested. I personally use it because its light one source, scalable, and very flexible. Have also chosen to use CoAP rather than REST for ad hoc data requests because for constrained devices it is much lighter on bandwidth, simpler to use than typical REST over TCP+HTTP with which most people are familiar. In IOT its hard to go past MQTT for the usual signals paired with CoAP for ad hoc. When I want to reduce dependencies or just being lazy even CoAP gets replaced with building a request/respond mechanism using a MQTT queue so a hub pubs a message to the devices command queue and device reads that queue and responds to it. How often device checks the command queue depends on whether or not device can run command queue monitoring asynchronously or must do a scheduled round-robin of the queues. The Eclipse Paho libs have been great for me using across different OS and bare metal, Arduino, FreeRTOS.
  - PostThisTooFast 5y ago
    Thanks. My application doesn't require ad-hoc queries; it's all about setting parameters on a device and getting status back.
jvanderbot 5y ago
I have used ZMQ for this with great success. The one-line python calls to send data between nodes are really nice.
- alias_neo 5y ago
  ZeroMQ is fantastic. I've been using at work for a few years now, but I generally enjoy the tech anyway, their documentation[0] and "The Guide"[1] is also really nice, with examples for all of the different language bindings. From C or Golang (or a bunch of other language bindings) you can set up some nice pub/sub or req/res with fan in/fan out and all sorts of cool routing with low-level, high speed sockets with the added power that ZMQ provides, such as the queueing, high-watermarks etc. You can also enable encryption easily with CurveZMQ which is a refreshing change from TLS. [0]https://zeromq.org/get-started/ https://zeromq.org/get-started/ [1]https://zguide.zeromq.org/ https://zguide.zeromq.org/
  - tombert 5y ago
    I almost feel like the guide should be mandatory reading for anyone learning network programming. It gives extremely digestible summaries of most of the network patterns, when and why you'd use them, and then how they're done in ZeroMQ. Even if you end the guide thinking that ZeroMQ is stupid (which you probably wouldn't), you'd still probably benefit from having read the guide.
- tombert 5y ago
  I love ZeroMQ, but I didn't think that had any first-class support for communicating in the browser (could be wrong). I agree it's super nice though, even with a single process.
RedShift1 5y ago
The VerneMQ bugtracker scares me. Especially this one: https://github.com/vernemq/vernemq/issues/1663 https://github.com/vernemq/vernemq/issues/1663. I'm running one instance but I'm on the lookout for an alternative that can more reliably save messages if a subscription client goes down.
- tombert 5y ago
  I wish I could say that I chose Verne.mq because I weighed all the options and it was the best for my project but...I really chose it because it was written in Erlang and I like Erlang. I won't pretend to have done an audit of the best servers in terms of bugs or features.
mosselman 5y ago
> I definitely recommend looking into trying MQTT before you jump straight into WebSockets. What is a nice way to use mqtt in the front-end instead of websockets? What can you recommend?
- oneweekwonder 5y ago
  Eclipse Paho JavaScript Client[0] for the front-end and personally I prefer RabbitMQ Web MQTT Plugin[1] for the back-end[1]. Take note Paho MQTT Client is available for several programming languages(Py,Rust,Go,C,C#,etc)[2]. [0]: https://www.eclipse.org/paho/index.php?page=clients/js/index.php https://www.eclipse.org/paho/index.php?page=clients/js/index... [1]: https://www.rabbitmq.com/web-mqtt.html https://www.rabbitmq.com/web-mqtt.html [2]: https://www.eclipse.org/paho/index.php?page=downloads.php https://www.eclipse.org/paho/index.php?page=downloads.php
  - tombert 5y ago
    Beat me to it! I was about to suggest the Eclipse Paho library. One of the very few things in the developer world that has "just worked" for me the first time I tried it.
deleted 5y ago
[deleted]
SirMonkey 5y ago
I learned about them on my previous job. It's rather easy to cluster and the Web-Hooks are super easy to get started with. I don't know Erlang myself, but looking at the code and comparing it to some of the Java solutions, it seemed well structured and easier to understand/modify if I wanted to. The only "Issue" I have with it right now, is the "custom OpenSource license" they have.

IceWreck 5y ago

I'm a big fan of NATS. It a small binary which can be scaled as much as you want and it replaces Kalfka, lightweight MQTT brokers, etc and has clients for most languages.

ducktective 5y ago
https://nats.io/ https://nats.io/ https://github.com/nats-io/nats-server https://github.com/nats-io/nats-server Is it suitable for embedded applications? (very low resources, intermittent network availability, etc)
- AkshitGarg 5y ago
  The NATS protocol [0] looks pretty simple, but is text based unlike MQTT which uses a size optimized binary protocol. [0]: https://docs.nats.io/nats-protocol/nats-protocol https://docs.nats.io/nats-protocol/nats-protocol
- DenseComet 5y ago
  NATS actually also has support for MQTT[0]. If you already have a deployment for queues, microservices, etc, using the NATS mqtt endpoint seems like a very good option. [0] https://docs.nats.io/nats-server/configuration/mqtt https://docs.nats.io/nats-server/configuration/mqtt

pletnes 5y ago

I take it mqtt was developed for embedded systems, microcontrollers etc to be lightweight. Can the broker also run in such and environment, or is that just the clients?

zorr 5y ago
Mosquitto is C/C++ without any crazy dependencies so you should be able to do that. I don't immediately see why you would do this though? Any specific use cases? I haven't done this myself yet but I've seen a couple of production IoT setups with a local MQTT broker running on a RaspberryPi configured to relay all messages to a central broker running in the cloud.
- jvanderbot 5y ago
  Pi is a far cry from "Microcontrollers". MQTT requires a TCP stack, which limits to the higher end chips. But, for embedded linux, you're probably golden.
  - joezydeco 5y ago
    MQTT is available via LWIP on STM32 parts that don't have an MMU. I've tried it on the STM32F746 (Cortex-M7) and it works well, although there are other issues with networking on CubeMX that are open.
detaro 5y ago
If you don't need it to scale crazily, yes. E.g. microcontrollers in nodes and then a local gateway as an embedded Linux system would not surprise me to see.
joezydeco 5y ago
I'm running Mosquitto as the central messaging broker on an embedded Linux product right now. It's super lightweight. Running the broker on the target has been a really great thing during development. I can run validation systems on other machines and inject messages over the wire, or I can actually run the UI on my dev machine and let the backend respond from a live target. I also have a development team in Asia that can't get to the newest hardware, so I fire up a system here and open a port they can access. They love it. That said, the bandwidth in MQTT is lightweight but you still need a TCP/IP stack on your system. That's typically more substantial of a footprint than Mosquitto will be at the end of the day.

deknos 5y ago

is mosquitto clusterable nowadays? on its own or via k8s? for: * fault tolerance * fail over * vertical scalability? does anyone know this? if not is there a another opensource mqtt broker which does that?

mlhales 5y ago
Mosquitto is not, we use RabbitMQ which runs on Erlang/OTP and has mqtt plug-ins. Mosquitto is single threaded. It's fast for sure but doesn't scale out. They can interoperate though as Mosquitto has configurable bridging.
eska 5y ago
There are broker implementations that support clustering. Emqx comes to mind.
serial_dev 5y ago
VerneMQ [0] could be interesting for you, they claim that: > VerneMQ is a high-performance, distributed MQTT broker. It scales horizontally and vertically on commodity hardware to support a high number of concurrent publishers and consumers while maintaining low latency and fault tolerance. I'm not sure about their licensing at the moment [1], to me it's a bit confusing. [0] https://vernemq.com/ https://vernemq.com/ [1] https://vernemq.com/blog/2019/11/26/vernemq-end-user-license-agreement.html https://vernemq.com/blog/2019/11/26/vernemq-end-user-license...
jonquark 5y ago
There are lots of brokers and Eclipse Amlen is coming soon: https://www.eclipse.org/amlen/ https://www.eclipse.org/amlen/ (disclosure: I work on Amlen)
ralight 5y ago
I work for Cedalo AG developing Mosquitto. Cedalo will be offering a version of Mosquitto with failover support as a paid for feature soon.
oneweekwonder 5y ago
Recently wanted to also create a cluster of mqtt brokers and found this medium tutorial stringing Docker Swarm, RabbitMQ, Consul and HAProxy together very helpful[0]. I was not happy with the static ip's for haproxy and opted to register the mqtt/tls services in consul and use the consul dns resolver with hapraxy. [0]: https://medium.com/hepsiburadatech/implementing-highly-available-rabbitmq-cluster-on-docker-swarm-using-consul-based-discovery-45c4e7919634 https://medium.com/hepsiburadatech/implementing-highly-avail...

zerkten 5y ago

For someone that has worked with RabbitMQ, is there a guide to the differences and appropriate usage of MQTT brokers with some real use cases explained?

oneweekwonder 5y ago
You are aware RabbitMQ support MQTT[0] and MQTT over WebSockets[1]? As for comparing it to RabbitMQ you should compare the AMQP protocol vs MQTT protocol, it seems MQTT with its lightweight design "won" IoT market over and should be rather compared against protocols like CoAP. As for guides I have always found resources rather scarce and mostly learned by adding features to mqtt libraries or writing my own prototype games using it. [0]: https://www.rabbitmq.com/mqtt.html https://www.rabbitmq.com/mqtt.html [1]: https://www.rabbitmq.com/web-mqtt.html https://www.rabbitmq.com/web-mqtt.html
- growse 5y ago
  Unfortunately, Rabbit's implementation of MQTT has some pretty serious deficiencies which you may find ends up being a deal-braker (e.g. https://github.com/rabbitmq/rabbitmq-server/issues/2556 https://github.com/rabbitmq/rabbitmq-server/issues/2556).

EastOfTruth 5y ago

I used Mosquitto years ago for a few years and never had any issues with it...

zwieback 5y ago

I like MQTT and we use Mosquitto in our product but I always wonder if MQTT pushes TCP into areas where UDP would be a better fit.

mlaretallack 5y ago
That is where MQTT-SN comes in. I have used it over NB-IOT and then a gateway to full MQTT over mTLS to AWS IoT.
ralight 5y ago
Like your other reply said, MQTT-SN can use UDP so may be a better fit in some situations. It's a shame that MQTT-SN hasn't seen wider use, but there isn't that much support at the moment. The protocol is being standardised at the moment, and there has been an MQTT to MQTT-SN gateway as part of the Eclipse Paho project for a while now. The Mosquitto project has the aim of adding support for MQTT-SN - it's just not as high a priority as other things at the moment unfortunately though. If it's something that's important to you, please get in touch. Roger

clumsysmurf 5y ago

Anyone have experience comparing MQTT with NATS, they seem to cover similar scenarios.

JesseObrien 5y ago
Would be looking for this as well. I love using NATS, would like to see a comparison of both.

eska 5y ago

Unfortunately the websockets implementation only allows IPv6, and in my tests from a js app sometimes took seconds to start if it worked at all. I’m currently looking for an alternative broker due to those issues.

oauea 5y ago
> Unfortunately the websockets implementation only allows IPv6 Huh? Pretty sure that's not true.
- eska 5y ago
  >socket_domain [ ipv4 | ipv6 ] >By default, a listener will attempt to listen on all supported IP protocol versions. >If you do not have an IPv4 or IPv6 interface you may wish to disable support for either of those protocol versions. In particular, note that due to the limitations of the websockets library, it will only ever attempt to open IPv6 sockets if IPv6 support is compiled in, and so will fail if IPv6 is not available. From the manual. Huh, seems like I might’ve misunderstood this paragraph. But I was only ever able to access the broker if the browser was using an ipv6 address. I’ll have to test again I suppose. Still, the performance and reliability was not good.
  - oauea 5y ago
    Right, that just says if you don't have IPv6 support compiled in, you can't use IPv6. IPv4 is always available, and has worked wonderfully well in my experience.
    - eska 5y ago
      Thanks for pointing that out. I’ll have to try again. Maybe ipv4 got disabled in the configuration somehow by accident.
ralight 5y ago
As others have pointed out, that's not the case. Something you do have to be careful of is the compile options for libwebsockets and what version you're using. Recent-ish version of libwebsockets in particular disable by default some features Mosquitto relies on, so some installations may suffer. Version 2.1 of Mosquitto will no longer need libwebsockets (although it can still be selected at compile time) and so websockets support will be less dependant on how other libraries are compiled.

rhodin 5y ago

We run a hosted version starting at $5/month [0]. We used to have a free version but it was too popular. [0] https://www.cloudmqtt.com/plans.html https://www.cloudmqtt.com/plans.html

jshier 5y ago
The free version was a great resource while developing MQTT clients a few years ago, sorry it was too expensive.
- rhodin 5y ago
  If you are a library developer/maintainer we'd be happy to sponsor you with free subscriptions, contact us at support@cloudmqtt.com.
jayzym 5y ago
We use this for work and it's a pretty slick product. Please add support for MQTT 5.0, or shared subscriptions!
pmx 5y ago
We use cloudmqtt at work and we've been really impressed by the price and quality of the product. You made it not worth hosting our own instance!
jasonvorhe 5y ago
In case someone's looking for a free MQTT service with the only constraints being client connection limits and bandwidth, check out HiveMQ Cloud: https://hivemq.cloud https://hivemq.cloud

Kuinox 5y ago

I'm developing a mqtt client/broker and the protocol is all but simple to implement. I didn't found any implementation, mosquitto included, that was really lossless.

RedShift1 5y ago
The protocol is simple (I've written my own implementation in php), what do you mean with lossless?
- Kuinox 5y ago
  That when you use QoS 2 "ExactlyOnce" (or 1), you will never loose a message.
luma 5y ago
Is that true if you set QoS to a value other than 0 in your application? https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels/ https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-qual...
- Kuinox 5y ago
  No, even when you set the QoS to 2, "ExactlyOnce", you may still hit edge cases where you loose data, without knowing it, especially on enterprise networks, or if there is an high throughtput.
xnyan 5y ago
I've worked quite a bit with MQTT - can you tell us a little more about what you mean by "really lossless"?
- Kuinox 5y ago
  To never loose a packet on QoS 1/2.
nicolaslem 5y ago

nicolaslem 5y ago

MQTT is becoming the defacto interface for home automation. As an example, Home Assistant used to have plugins for zwave and zigbee but they are now replaced by an MQTT broker that sits between Home Assistant and the program driving the wireless dongle. Since I'm already running Mosquitto to interface with zwave devices, I decided to also use it to monitor my home lab. Servers push sensor statuses into Mosquitto and Home Assistant is used for the UI and alerting. That was a fun week-end project.

oynqr 5y ago
ZHA is still the preferred way for Zigbee with Home Assistant, isn't it?
- EngCanMan 5y ago
  Many people use https://www.zigbee2mqtt.io/ https://www.zigbee2mqtt.io/ now too, it seems people are moving in this direction
- dusing 5y ago
  ZHA is great and fast, I prefer it instead of zigbee2mgtt. I have 100+ zigbee devices and 80 z-wave.
- nathan_f77 5y ago
  I started with ZHA but had problems with some devices (e.g. a 4-way switch panel that would only show as a single button.) I've had a much better experience with zigbee2mqtt, and it seems a bit more powerful
- specto 5y ago
  Saw this too recently - https://www.cnx-software.com/2021/09/27/zb-gw03-esp32-based-ethernet-zigbee-gateway-now-working-with-tasmota-firmware/ https://www.cnx-software.com/2021/09/27/zb-gw03-esp32-based-..., quite nice
- jhot 5y ago
  I've been happy with the ZHA implementation and haven't tried z2mqtt. I am much happier with zwavejs than the old implementation, though. I do run MQTT for a few wifi devices and communication between node red and custom client code on my laptop/desktop.

SavantIdiot 5y ago

Anyone have experience scaling with more than 256 devices? I use MQTTS at work and for my own projects. It is a very thin protocol (low code footprint), especially combined with Wi-Fi where those stacks typically eat up most of your MCU's flash if you're using a single-chip solution with TLS. I've used it on a max of 256 devices on a single BSSID, but I would be curious to know how it scales. EDIT: Specifically, when all devices are subbed to a noisy channel. In some cases we had to stagger pub with random jitter to prevent both Wi-Fi and edge processing issues.

eska 5y ago
I’ve only seen papers from the automobile industry where they used MQTT but fed it into Kafka to scale.
IshKebab 5y ago
All Dyson smart products use MQTT. As I recall it works fine from a performance point of view (it's a very lightweight protocol), but personally I thought it was a pretty flawed choice because the problem of choosing/discovering paths is just completely ignored, and it shares the same problem as all event bus style systems - events can get lost, it's very difficult to debug and learn, etc.
- SavantIdiot 5y ago
  > events can get lost, Well, yes, but QoS=1 or 2 with a reasonably large queue I haven't seen loss issues. What kind of loss problems are you running into (that I should be worried about)?
  - IshKebab 5y ago
    Yeah I don't mean due to the protocol, I mean due to application bugs where something adds a message to some queue and the thing that is meant to process it doesn't for some reason, and then nothing will ever realise. Compared to callbacks / RPC for example, where you have a function call the either has to succeed or it has to fail. It's not a deal breaker but I don't like it.

mleo 5y ago

MQTT/Mosquitto relays is a great solution for passing data across network boundaries when inbound is limited. I use it with Webhooks deployed to Internet VM that pass the data to broker installed on VM. Another broker in home network connects to Internet broker and relays messages to local clients. I don’t have to open any ports into my home network for webhooks to work with other services.

jvanderbot 5y ago

I've recently transitioned my home "IOT" network (and servers) to a Wireguard-only network. It's wonderful. All the routing accessibility through firewalls or with dynamic IPs just disappeared. All services are exposed only on the wireguard interface, meaning security and connectivity are solved (to the 80/20 level) with just Wireguard. That kind of removed the need for a static broker entirely since now every node has a static IP on my wireguard network and are accessible no matter what network they are actually on. Pinging my mobile phone from an air quality sensor was an eye-opening experience.

kobalsky 5y ago
what kind of IOT devices are we talking about? linux embedded or is wireguard available for something like nodemcu?
- jvanderbot 5y ago
  Linux embedded. (Pis mostly)
3np 5y ago
Did you ever notice any issues with significant increased softirqs? I started going down a similar path but a node that is acting as a gateway/bridge for UDP traffic routing to an outbound wireguard tunnel is now spending a constant ~20% CPU time on NET_RX softirqs with 10Mbps and 5~30Kpps. Almost no NET_TX. It's specifically the combination of UDP-over-wg that seems to cause this. It's otherwise handling 1Gbps and way more packets just fine. (I'm aware MQTT is TCP but hey, worth a shot)
jve 5y ago
Hey, tell us more, please :) I'v hoped to find in your profile maybe more info/blog, but instead: > Group supervisor at NASA Jet Propulsion Laboratory working in autonomous vehicles and sensing for space exploration. Oh well, a little out of scope and I see you blog about other things. Anyway - you mean I don't need static IP to form a network of distant IoT devices? This would be some really useful property.

travisgriggs 5y ago

I'll just pile on my own accolades. We use MQTT to connect handhelds with in the field ag irrigation controllers. There's probably about 10,000 valves on farms and other ag projects throughout, that are ultimately scheduled and monitored through MQTT. We've even been able to build a full request/event protocol on top of it. The protocol is simple enough, that when I wasn't happy with not many full solutions existing in iOS, I just rolled our own in a day or two. Currently adding an Elixir REST bridge to the whole thing. It's been pretty cool all around.

ralight 5y ago
That sounds really nice, I love interesting projects in areas outside of my experience. If you're using Mosquitto I'd love to hear more about it - my contact details are at the bottom of https://github.com/eclipse/mosquitto/ https://github.com/eclipse/mosquitto/ :)

bsmth 5y ago

I really like Mosquitto, I'd recommend it for quickly getting started hacking together IoT projects on the hobby / side project scale. One of our community (QuestDB) contributors put together a docker compose example that I thought was pretty nice for quick prototyping, could be convenient for someone here: https://github.com/shantanoo-desai/questitto https://github.com/shantanoo-desai/questitto

monkeydust 5y ago

Use Mosquitto on Windows server at home to transfer data from my energy smart meter to home assistant. Pretty rock solid and easy to diagnose issues via mqtt tools.

ajfally 5y ago

I've been wanting a GCM/FCM alternative, could mqtt be used for notifications like that? I understand its usually used for IOT devices and stuff so I assume its suitable for not using loads of power

malermeister 5y ago
Thats what Facebook Messenger uses. Downside to this is that only FCM can work around android's power saving mechanisms and deliver instantly... unless you're Facebook and able to convince vendors to patch the firmware for your servers too ;)
- holoduke 5y ago
  Yep. Another real concrete example of an unwanted monopoly. We have issues with fcm, we are sending about 10m per day for millions of users. About 1 to 3% never arrives. Ends up in the void. Google doesn't want to help. We are not Facebook. But no alternative. A pity
  - malermeister 5y ago
    97% is actually a really good delivery rate. From my experience working on several push-based products, ~80% is the industry standard.

nathan_f77 5y ago

I've learned a lot about MQTT while setting up Home Assistant, and Mosquitto is a pretty important part of my Home Assistant installation. I use it to control all of my Zigbee devices via "zigbee2mqtt" (buttons, switch modules, etc.), and also a smart kettle running Tasmota [1]. I also added a little patch to Tasmota [2] so that my kettle responds with `418 I'm a teapot` [3] [1] https://tasmota.github.io/docs https://tasmota.github.io/docs [2] https://github.com/ndbroadbent/Tasmota/commit/bbcf57faffbf5b7fab5c54f395b7b720f8df50b6 https://github.com/ndbroadbent/Tasmota/commit/bbcf57faffbf5b... [3] https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/418 https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/418

seidoger 5y ago
Prime use-case for HTTP 418!
- SV_BubbleTime 5y ago
  People always want that for teapots. I have a device that uses it as an indication it’s internal data shouldn’t be trusted (some unknown error). I have it respond with 418 as “I taste colors and the government spies on you with the ground in your house wiring”. I like it for that more than actual tea pots which is far too on the nose.
  - Something1234 5y ago
    What kind of device might have internal data that cannot be trusted?
- deleted 5y ago
  [deleted]
matttrotter 5y ago
Cool stuff and inspiring. Have a website or GitHub about your home automation projects? I checked out your blog, and the raspberry pi microwave was pretty sweet.

bborud 5y ago

This project is exceptionally useful not only in itself but as a tool you can use during development. Software that can be controlled entirely from the command line without configuration files (unless you choose to) are very convenient.

boilerupnc 5y ago

The node-red [0] project is a great complement to MQTT use-cases. [1] "Node-RED is a programming tool for wiring together hardware devices, APIs and online services in new and interesting ways. It provides a browser-based editor that makes it easy to wire together flows using the wide range of nodes in the palette that can be deployed to its runtime in a single-click." [0]: https://nodered.org/ https://nodered.org/ [1]: https://cookbook.nodered.org/mqtt/connect-to-broker https://cookbook.nodered.org/mqtt/connect-to-broker

deepspace 5y ago
Node-RED is one of those polarizing apps. People either love it or hate it with a passion. I am in the latter camp. I mean, it does what it says on the tin, but I could never understand how laboriously "wiring" up a bunch of boxes and filling in parameters is any easier than whipping up a few lines of Python.
- sleavey 5y ago
  I'm in your camp too, but would it be fair to say that Node-RED is not designed for us?
- EmpirePhoenix 5y ago
  Well, if you are not a code based developer, eg. you are used to program industrial control systems, they are often developed the same way, as with certified/proofed tools, you can guarantee that the code has no uncaught error paths or simple syntax errors. It also has a lot appeal to people who do not develop on a daily basis, eg. I'm pretty confident I could teach my dad to use nodered vs writing python in a day. I myself just use it (after years of being on the second side) because I noticed, that it was somehow way faster to get stuff working and keep it maintained
- slightwinder 5y ago
  Isn't it because the boxes have in-place documentation and the wires visualize the workflow? Making it easy & fast for the occasional user and let them avoid the pitfalls of unmastered syntax.

yasserf 5y ago

MQTT is awesome. In deepstream.io we bridged the node MQTT protocol to deepstream which allows us to mix and match protocols. MQQT for IoT devices (and to be honest any device considering the large amount of libraries) and the deepstream JS client for node/browsers. This pretty much allowed clustering out of the box + automatically saves messages with quality assurance set in an actual database. Plus a bit more authentication and permission tools. https://deepstream.io/blog/20190909-release-deepstream-4.2/ https://deepstream.io/blog/20190909-release-deepstream-4.2/ Deepstream itself is under maintenance right now, but I find the concept of mixing protocols together (similar to what ably.io provides) to hit a sweet spot for development across multiple different environments/devices

canada_dry 5y ago

Perhaps someone here that's familiar with MQTT (and variants) can shed light on how to handle this case: an IoT device that periodically comes to life, queries the broker for the last known state of topic 'xyz', then sends a message regarding another topic 'abc' and goes back to sleep. i.e. it's not awake to receive updates to the 'xyz' topic, but needs to know the last update when it awakes.

Kuinox 5y ago
With mqtt you may do that with retained messages. The emitter of a message mark the message as to be retained. The server will retain it. Now when your mqtt client will subscribe tout the topic, the server will send the last retained message for this topic.

jpgvm 5y ago

If you need a highly available broker you can consider https://github.com/streamnative/mop https://github.com/streamnative/mop which adds a MQTT protocol handler to Apache Pulsar. I recently contributed auth support and TLS support also just landed. There is perhaps a few non-standards compliant things that need to be worked out but it's in fairly good shape.

noisy_boy 5y ago

I think it is time to use Kafka for home automation messages.

adolph 5y ago
Enterprise-class home automation https://developer.ibm.com/tutorials/kafka-in-kubernetes/ https://developer.ibm.com/tutorials/kafka-in-kubernetes/

BrandoElFollito 5y ago

I started home automation in my late 40-ies, as an amateur developer. Despite having casually developed for 25 years at that point, I found it amazing how much I learned quickly when setting up Home Assistant and pyscript (and obviously mqtt). This drove me to use a bus (mqtt for instance) for non-home automation code just because it is so useful. To friends who wonder how to help their kids get interested in coding (which is not easy at the beginning because converting farenheit to celsius gets boring after some time), I tell them to buy a few zigbee devices and have them automate stuff in pyscript. I am happy to see that some are on the way to graduate in CS :)

randomluck040 5y ago

We’re using MQTT to have different sensors publish their measurements and different user groups can subscribe to the relevant ones. I think it’s a great way to separate stuff from each other or send meaningful data away together.

ralight 5y ago

This should have been my first comment, but there you go. Hello, I'm Roger, the Mosquitto project lead. Thanks for the kind comments, I'm happy to answer questions! I work for cedalo.com developing Mosquitto and other MQTT projects. We offer Mosquitto based support, please get in touch if that's of interest to you.

zorr 5y ago
Thank you for your work!
ciberado 5y ago
I just wanted to thank you for it. Over the years it has had a big impact on all types of projects, from industrial heavy ones to raspberry-based tinkering experiments :D

consp 5y ago

We recently used mosquitto/mqtt as the broker for a timekeeping system for a student engineering event at a track and was mirrored to the web/livestream for viewing pleasure of everyone else at home (since it was during the pandemic). It worked with minimal load and without problems even when the connection was shoddy due to extremely badly configured routers by vodafone (filling up nat tables is still a thing in 2021 apperantly).

Hema0909 5y ago

I really like Bevywise MQTTRoute[1] as it is good one for home automation solutions. It allows us to create our dashboard[2] specific to our use case with pre-built widgets. The easy & flexible integration with home assistant[3] and the mobile app[4] permits us to automate our home devices. [1] https://www.bevywise.com/mqtt-broker/ https://www.bevywise.com/mqtt-broker/ [2] https://www.bevywise.com/mqtt-dashboard/ https://www.bevywise.com/mqtt-dashboard/ [3] https://www.bevywise.com/blog/mqtt-route-integration-with-home-assistant/ https://www.bevywise.com/blog/mqtt-route-integration-with-ho... [4] https://play.google.com/store/apps/details?id=com.Bevywise.mqttroute https://play.google.com/store/apps/details?id=com.Bevywise.m...