3 ms·

I expected to see MQTT as an integral part of the WebThings Gateway but haven't found it so far

by pagutierrezn 7y ago

IshKebab 7y ago
MQTT is kind of crap so I'm not surprised.
- Uberphallus 7y ago
  What's the problem with MQTT? It's basically the interconnecting standard for all things IoT. I mix and match IoT devices, and everything is fed into Home Assistant through MQTT, plus logging to InfluxDB. I'm familiar with the protocol and haven't found any wart; the biggest problem is how to run it with redundancy and at a scale.
  - kbumsik 7y ago
    AFAIK there is no IoT data exchange format standard over MQTT. Every vendors has its own data format. It is not simple to normalize the data to match other standards like Zigbee profiles. You should manually code for every MQTT devices.
  - musingsole 7y ago
    I cannot stand debugging MQTT devices. From establishing a connection to sending data, it is consistently difficult to pinpoint where the failure in a system is occuring. Contrast this with HTTP and WebSockets that are extremely simplistic to use. Largely by virtue of bigger communities using them, but so it is. Where's the benefit of MQTT? The code is just as large and more opaque by being an uncommon standard (compared to HTTP). Everyone says bandwidth, but that's hardly a concern often and when it is...I can never seem to find the numbers of how much data usage MQTT would save for a given application.
  - Matthias247 7y ago
    Some issues that come to my mind with MQTT: 1. Most implementations are quite weak. E.g. the Paho implementation for C can't send and receive at the same time, can't publish multiple messages concurrently, and will block it's only execution thread for lots of seconds until the server might response. That might be ok for very low-throughput devices, but in general it's neither performant nor resilient. Libraries like these also don't track whether subscriptions already have been performed or not, don't perform retries, etc. 2. The specification has a few warts, that imho make things like QoS1 and above impossible to implement correctly. E.g. if a message is sent with a certain messageId and the server doesn't respond within a timeframe, the client can't really time-out only this publication. If the client forgets the messageId, continues to work and later on publishes a new message with the same Id (there are only 64k of those), then any ACK for that is ambiguous and the client couldn't tell for sure which delivery was ACKd. That basically voids the at least once delivery property. I don't any implementation out there blacklists the IDs in order to avoid the ambiguity (which would also not work long term anyway, due to the low address space). 3. Most applications want some kind of transactional or request/response semantics. Of course that can be implemented on top of MQTT in the application layer, but it's more tedious and error-prone than using a protocol which already enables it. 4. Lots of applications also want to stream bigger amounts of data for some use-cases. Implementing a multi-megabyte upload or download with MQTT requires to build a custom flow-control, retransmissions, buffering for out-of-order messages, etc. Or in other words: They are required to implement the equivalent of TCP on top of UDP. Overall my opinion is that it can be used for certain applications, but it's actually a far weaker option than assumed. Due to the issue 1) and 2) I would recommend to mainly use QoS0 if talking to a MQTT broker is a requirement - and to move all reliability concerns into the application layer instead of relying on MQTT doing it right.

I expected to see MQTT as an integral part of the WebThings Gateway but haven't found it so far

IshKebab 7y ago

MQTT is kind of crap so I'm not surprised.

Uberphallus 7y ago
What's the problem with MQTT? It's basically the interconnecting standard for all things IoT. I mix and match IoT devices, and everything is fed into Home Assistant through MQTT, plus logging to InfluxDB. I'm familiar with the protocol and haven't found any wart; the biggest problem is how to run it with redundancy and at a scale.
- kbumsik 7y ago
  AFAIK there is no IoT data exchange format standard over MQTT. Every vendors has its own data format. It is not simple to normalize the data to match other standards like Zigbee profiles. You should manually code for every MQTT devices.
- musingsole 7y ago
  I cannot stand debugging MQTT devices. From establishing a connection to sending data, it is consistently difficult to pinpoint where the failure in a system is occuring. Contrast this with HTTP and WebSockets that are extremely simplistic to use. Largely by virtue of bigger communities using them, but so it is. Where's the benefit of MQTT? The code is just as large and more opaque by being an uncommon standard (compared to HTTP). Everyone says bandwidth, but that's hardly a concern often and when it is...I can never seem to find the numbers of how much data usage MQTT would save for a given application.
- Matthias247 7y ago
  Some issues that come to my mind with MQTT: 1. Most implementations are quite weak. E.g. the Paho implementation for C can't send and receive at the same time, can't publish multiple messages concurrently, and will block it's only execution thread for lots of seconds until the server might response. That might be ok for very low-throughput devices, but in general it's neither performant nor resilient. Libraries like these also don't track whether subscriptions already have been performed or not, don't perform retries, etc. 2. The specification has a few warts, that imho make things like QoS1 and above impossible to implement correctly. E.g. if a message is sent with a certain messageId and the server doesn't respond within a timeframe, the client can't really time-out only this publication. If the client forgets the messageId, continues to work and later on publishes a new message with the same Id (there are only 64k of those), then any ACK for that is ambiguous and the client couldn't tell for sure which delivery was ACKd. That basically voids the at least once delivery property. I don't any implementation out there blacklists the IDs in order to avoid the ambiguity (which would also not work long term anyway, due to the low address space). 3. Most applications want some kind of transactional or request/response semantics. Of course that can be implemented on top of MQTT in the application layer, but it's more tedious and error-prone than using a protocol which already enables it. 4. Lots of applications also want to stream bigger amounts of data for some use-cases. Implementing a multi-megabyte upload or download with MQTT requires to build a custom flow-control, retransmissions, buffering for out-of-order messages, etc. Or in other words: They are required to implement the equivalent of TCP on top of UDP. Overall my opinion is that it can be used for certain applications, but it's actually a far weaker option than assumed. Due to the issue 1) and 2) I would recommend to mainly use QoS0 if talking to a MQTT broker is a requirement - and to move all reliability concerns into the application layer instead of relying on MQTT doing it right.