3 ms·

Anyone have experience scaling with more than 256 devices? I use MQTTS at work and for my own projects. It is a very thin protocol (low code footprint), especi

Anyone have experience scaling with more than 256 devices? I use MQTTS at work and for my own projects. It is a very thin protocol (low code footprint), especially combined with Wi-Fi where those stacks typically eat up most of your MCU's flash if you're using a single-chip solution with TLS. I've used it on a max of 256 devices on a single BSSID, but I would be curious to know how it scales. EDIT: Specifically, when all devices are subbed to a noisy channel. In some cases we had to stagger pub with random jitter to prevent both Wi-Fi and edge processing issues.

eska 5y ago
I’ve only seen papers from the automobile industry where they used MQTT but fed it into Kafka to scale.
IshKebab 5y ago
All Dyson smart products use MQTT. As I recall it works fine from a performance point of view (it's a very lightweight protocol), but personally I thought it was a pretty flawed choice because the problem of choosing/discovering paths is just completely ignored, and it shares the same problem as all event bus style systems - events can get lost, it's very difficult to debug and learn, etc.
- SavantIdiot 5y ago
  > events can get lost, Well, yes, but QoS=1 or 2 with a reasonably large queue I haven't seen loss issues. What kind of loss problems are you running into (that I should be worried about)?
  - IshKebab 5y ago
    Yeah I don't mean due to the protocol, I mean due to application bugs where something adds a message to some queue and the thing that is meant to process it doesn't for some reason, and then nothing will ever realise. Compared to callbacks / RPC for example, where you have a function call the either has to succeed or it has to fail. It's not a deal breaker but I don't like it.
    - RedShift1 5y ago
      If your MQTT client allows control over this aspect of the protocol, you can just not ACK QOS > 0 messages that failed to process. Next time you connect you'll get the non-ACK'ed messages first.
lulf 5y ago
You can use a subset of MQTT for ingress data and store it in Kafka for instance. Check out https://www.drogue.io https://www.drogue.io and https://sandbox.drogue.cloud https://sandbox.drogue.cloud for an open source project that does that
ralight 5y ago
I can't comment on MCUs or WiFi BSSIDs, but on the server side I have an Intel Atom N2800 (think low performance) with nearly 4000 clients sending out ~50k messages/s with an average payload size of 80 bytes. The overwhelming majority of those are not using TLS. About a quarter using MQTT over websockets.

eska 5y ago

I’ve only seen papers from the automobile industry where they used MQTT but fed it into Kafka to scale.

IshKebab 5y ago

All Dyson smart products use MQTT. As I recall it works fine from a performance point of view (it's a very lightweight protocol), but personally I thought it was a pretty flawed choice because the problem of choosing/discovering paths is just completely ignored, and it shares the same problem as all event bus style systems - events can get lost, it's very difficult to debug and learn, etc.

SavantIdiot 5y ago
> events can get lost, Well, yes, but QoS=1 or 2 with a reasonably large queue I haven't seen loss issues. What kind of loss problems are you running into (that I should be worried about)?
- IshKebab 5y ago
  Yeah I don't mean due to the protocol, I mean due to application bugs where something adds a message to some queue and the thing that is meant to process it doesn't for some reason, and then nothing will ever realise. Compared to callbacks / RPC for example, where you have a function call the either has to succeed or it has to fail. It's not a deal breaker but I don't like it.
  - RedShift1 5y ago
    If your MQTT client allows control over this aspect of the protocol, you can just not ACK QOS > 0 messages that failed to process. Next time you connect you'll get the non-ACK'ed messages first.

lulf 5y ago

You can use a subset of MQTT for ingress data and store it in Kafka for instance. Check out https://www.drogue.io https://www.drogue.io and https://sandbox.drogue.cloud https://sandbox.drogue.cloud for an open source project that does that

ralight 5y ago

I can't comment on MCUs or WiFi BSSIDs, but on the server side I have an Intel Atom N2800 (think low performance) with nearly 4000 clients sending out ~50k messages/s with an average payload size of 80 bytes. The overwhelming majority of those are not using TLS. About a quarter using MQTT over websockets.