8 ms·

Benchmarking latency across common wireless links for microcontrollers

135 points by killcoder 3y ago

killcoder 3y ago
We nerd sniped ourselves into testing the latencies of a whole bunch of wireless communication links and protocols for microcontrollers. We’ll probably do a series of power consumption / range tests later on, let me know if there are any setups in particular that you’d be interested in seeing test cases for. Raw data, firmware and post processing scripts are here on GitHub: https://github.com/Scottapotamas/embedded-wireless-latency-eval/ https://github.com/Scottapotamas/embedded-wireless-latency-e...
- deleted 3y ago
  [deleted]
- pockybum522 3y ago
  Once there's decent embedded HaLow options, I would love to see analysis of this level on it.
  - scottapotamas 3y ago
    I nearly did, but the write-up was getting pretty long. I'll try to find something for the planned range/interference tests. Morse Micro is also an Australian company so I'll probably look into their parts first unless there's any recommendation?
    - pockybum522 3y ago
      That sounds good if you're an Aussie. I'm honestly confused why there's still so few options, but I guess most radically new standards got a comparatively slow start.
pockybum522 3y ago
This is one of the most thorough and well laid out analysis of useful hobbyist and low level research options out there. I'll be referencing this for years, thank you very much. Clearly a lot of effort has been put into this.
jauntywundrkind 3y ago
If we had more great reviews like this we'd see hardware improve faster! Thank you! Shine that light!
comboy 3y ago
Outstanding blog post. NRF24 had been my fav for a long time, I planned switching to LoRa but I didn't know the latency is that high.
- steve_gh 3y ago
  You go for LoRa or NBIoT when you need the low power consumption, and especially for deep penetration. Generally, if you have a better option, you use it. LoRa and similar high latency networks can't easily run TCP due to the standard TCP timers (especially Ack timeouts). What I would like to see is better support from the cloud providers for UDP based protocols such as MQTT-SN.
zh3 3y ago
That is great stuff, many thanks for it. While not in such depth, for a precision timing network covering multiple sensors our analysis also showed that the delays and variation inherent in most higher-level radio protocols are far too high for microsecond-level synchronisation. For this reason, we use the nRF52 radio peripheral directly (i.e. barebones C) which means we know exactly when the packet was sent and - at the other end - when it was received. Details are in the Production Specification document, which also shows how long it takes for the transmit to spin up (turn on, transmit pre-amble and so on).
skywal_l 3y ago
Something of a tangent but I couldn't find anywhere a way to wirelessly extend usb ports. Allowing you to plug a mouse/keyboard/Yubikey to a computer in another room with minimum latency and without the need of any software or particular drivers. If one wanted to do something like that one would probably start by reading that article I guess. Thanks for this very thorough work.
- zamadatix 3y ago
  WiGig docks were promoted for a short while but the problem they ran into was having an e.g. keyboard in another room wasn't very practical unless the display was as well and that bumped the RF requirements. So it ended up in the 60 GHz range which wouldn't make it to the other room and people basically said "eh, what's the point I'll just plug the dock in and it works".
- justsomehnguy 3y ago
  USBAnywhere + any WiFi AP You would need drivers on the computer and I don't know about latency.
- byb 3y ago
  I've had the same desire to extend USB HID devices, but I want to take it a step further. Once a USB device <- Wi-Fi ->USB host technology is available, then a wireless USB switch should be possible. The research in this article is critical to understanding the impact of latency on the experience. Several months ago, I built a project using USB host mode on a Raspberry Pi Pico, which uses a USB numpad to pass messages to an MQTT server. This uses the Rp2040's USB host mode https://gitlab.com/baiyibai/pico-w-usb-host-mqtt-numpad https://gitlab.com/baiyibai/pico-w-usb-host-mqtt-numpad https://github.com/adafruit/Adafruit_TinyUSB_Arduino https://github.com/adafruit/Adafruit_TinyUSB_Arduino My implementation plugs in much higher into the stack and doesn't read the full USB bit mask output, which indicates how many keys are pressed/released. From my limited understand ing, it should be possible to pass these USB messages from HID client to HID host and vice-versa. Then it's only a matter of sending this information over a TCP/UDP interface. However, from my research, it seems the TinyUSB library doesn't provide the full bitmask resolution necessary for all devices. The YouTuber Wendell from LevelOneTechs has also talked about the troubles of getting some devices working with his KVM products, so it may not be as simple as I'm suggesting here. Overall though, a $15/endpoint is very attractive.
snovv_crash 3y ago
Super cool. I'd love to see 2.4GHz LoRA as well.
- andai 3y ago
  Not a radio guy but, isn't the whole point of LoRa to use lower frequencies for greater range?
  - snovv_crash 3y ago
    It also uses a different modulation so isn't as affected by interference from the more common 2.4GHz sources, while 2.4 gives you a lot more bandwidth and makes licensing easier.
davidw 3y ago
Years ago I was working on a system where an Android device communicated with a web server written in Erlang, and our Android guy was going off about how Erlang was "really slow" and that made no sense to me. So we looked at the interaction times, and it was certainly true that something was slow. Instead of the Erlang server, we swapped out Apache serving a static page. That was slow too. It turned out he'd been benchmarking the crappy wifi connection.
Franscoben 3y ago
[dead]
utopcell 3y ago
Great article! Here's a perspective from a guitarist's viewpoint. Most wireless guitar systems boast latencies below 3ms, with the better ones being closer to 1ms. Latency is important here because even small delays can be felt by the player. Since sound travels at ~1 ft per 1ms, a delay on the order of BLE is equivalent to playing with the amplifier ~25ft away from you, which is pretty bad. It is surprising that the best system is actually nrf24, given how old it is (or maybe because of that ?). It also seems to have enough bandwidth to transfer a 24-bit/48kHz signal.
- polishdude20 3y ago
  With guitar, how would the latency stack up when you add a/d conversion?
  - zh3 3y ago
    A/D conversion itself wouldn't add much at the sample level (say 48KHz sample, it's about 50us per sample). However, packetisation will - a 256 byte packet of 128 samples is 128*50us = 6.4ms right there at the transmitter, and the receiver won't notify until the full packet is received. So a naive digital approach would be 12.8ms (2x6.4ms) even before anything else. A pure analogue approach (modulated RF) on the other hand shouldn't have any human-detectable delay - it's effectively distance/speed of light with a bit of a phase shift (addtional delay) introduced by the electronics - should be only a handful of microseconds in total.
    - polishdude20 3y ago
      Analogue would be nice in that regard but wouldn't it be pretty bad with interference, signal quality over distances etc?
      - zh3 3y ago
        It worked that like for many, many years (radio mics) before digital came along. Think of FM radio - if you have a good signal, it's pretty resilient to interference and in a controlled short range environment it is extremely reliable.
nsasch 3y ago
Thanks for sharing your research! I’ve done some similar, but not as thorough, tests with 2.4ghz WiFi and 915MHz LoRa before. My goal is to sync time across multiple devices that go in and out of range of each other to play 60fps light animations and sound within half a frame of each other (8ms), with spare time for some computation. I’ve been surprised at how bad some oscillators (or voltage regulators) can be and their effect on consistent latency. I was having fun experimenting, but this will be really useful to get me back to actually implementing my project.
nanomonkey 3y ago
This is a great breakdown, although it doesn't go into any detail on the distance trade offs of each transmission type, which is an important part of the comparison. I look forward to when they go into this part of the analysis.
noman-land 3y ago
Amazing article and research. Most definitely bookmarked.

135 points by killcoder 3y ago

We nerd sniped ourselves into testing the latencies of a whole bunch of wireless communication links and protocols for microcontrollers. We’ll probably do a series of power consumption / range tests later on, let me know if there are any setups in particular that you’d be interested in seeing test cases for. Raw data, firmware and post processing scripts are here on GitHub: https://github.com/Scottapotamas/embedded-wireless-latency-eval/ https://github.com/Scottapotamas/embedded-wireless-latency-e...

deleted 3y ago
[deleted]
pockybum522 3y ago
Once there's decent embedded HaLow options, I would love to see analysis of this level on it.
- scottapotamas 3y ago
  I nearly did, but the write-up was getting pretty long. I'll try to find something for the planned range/interference tests. Morse Micro is also an Australian company so I'll probably look into their parts first unless there's any recommendation?
  - pockybum522 3y ago
    That sounds good if you're an Aussie. I'm honestly confused why there's still so few options, but I guess most radically new standards got a comparatively slow start.

pockybum522 3y ago

This is one of the most thorough and well laid out analysis of useful hobbyist and low level research options out there. I'll be referencing this for years, thank you very much. Clearly a lot of effort has been put into this.

jauntywundrkind 3y ago

If we had more great reviews like this we'd see hardware improve faster! Thank you! Shine that light!

comboy 3y ago

Outstanding blog post. NRF24 had been my fav for a long time, I planned switching to LoRa but I didn't know the latency is that high.

steve_gh 3y ago
You go for LoRa or NBIoT when you need the low power consumption, and especially for deep penetration. Generally, if you have a better option, you use it. LoRa and similar high latency networks can't easily run TCP due to the standard TCP timers (especially Ack timeouts). What I would like to see is better support from the cloud providers for UDP based protocols such as MQTT-SN.

zh3 3y ago

That is great stuff, many thanks for it. While not in such depth, for a precision timing network covering multiple sensors our analysis also showed that the delays and variation inherent in most higher-level radio protocols are far too high for microsecond-level synchronisation. For this reason, we use the nRF52 radio peripheral directly (i.e. barebones C) which means we know exactly when the packet was sent and - at the other end - when it was received. Details are in the Production Specification document, which also shows how long it takes for the transmit to spin up (turn on, transmit pre-amble and so on).

skywal_l 3y ago

Something of a tangent but I couldn't find anywhere a way to wirelessly extend usb ports. Allowing you to plug a mouse/keyboard/Yubikey to a computer in another room with minimum latency and without the need of any software or particular drivers. If one wanted to do something like that one would probably start by reading that article I guess. Thanks for this very thorough work.

zamadatix 3y ago
WiGig docks were promoted for a short while but the problem they ran into was having an e.g. keyboard in another room wasn't very practical unless the display was as well and that bumped the RF requirements. So it ended up in the 60 GHz range which wouldn't make it to the other room and people basically said "eh, what's the point I'll just plug the dock in and it works".
justsomehnguy 3y ago
USBAnywhere + any WiFi AP You would need drivers on the computer and I don't know about latency.
byb 3y ago
I've had the same desire to extend USB HID devices, but I want to take it a step further. Once a USB device <- Wi-Fi ->USB host technology is available, then a wireless USB switch should be possible. The research in this article is critical to understanding the impact of latency on the experience. Several months ago, I built a project using USB host mode on a Raspberry Pi Pico, which uses a USB numpad to pass messages to an MQTT server. This uses the Rp2040's USB host mode https://gitlab.com/baiyibai/pico-w-usb-host-mqtt-numpad https://gitlab.com/baiyibai/pico-w-usb-host-mqtt-numpad https://github.com/adafruit/Adafruit_TinyUSB_Arduino https://github.com/adafruit/Adafruit_TinyUSB_Arduino My implementation plugs in much higher into the stack and doesn't read the full USB bit mask output, which indicates how many keys are pressed/released. From my limited understand ing, it should be possible to pass these USB messages from HID client to HID host and vice-versa. Then it's only a matter of sending this information over a TCP/UDP interface. However, from my research, it seems the TinyUSB library doesn't provide the full bitmask resolution necessary for all devices. The YouTuber Wendell from LevelOneTechs has also talked about the troubles of getting some devices working with his KVM products, so it may not be as simple as I'm suggesting here. Overall though, a $15/endpoint is very attractive.

snovv_crash 3y ago

Super cool. I'd love to see 2.4GHz LoRA as well.

andai 3y ago
Not a radio guy but, isn't the whole point of LoRa to use lower frequencies for greater range?
- snovv_crash 3y ago
  It also uses a different modulation so isn't as affected by interference from the more common 2.4GHz sources, while 2.4 gives you a lot more bandwidth and makes licensing easier.

davidw 3y ago

Years ago I was working on a system where an Android device communicated with a web server written in Erlang, and our Android guy was going off about how Erlang was "really slow" and that made no sense to me. So we looked at the interaction times, and it was certainly true that something was slow. Instead of the Erlang server, we swapped out Apache serving a static page. That was slow too. It turned out he'd been benchmarking the crappy wifi connection.

Franscoben 3y ago

[dead]

utopcell 3y ago

Great article! Here's a perspective from a guitarist's viewpoint. Most wireless guitar systems boast latencies below 3ms, with the better ones being closer to 1ms. Latency is important here because even small delays can be felt by the player. Since sound travels at ~1 ft per 1ms, a delay on the order of BLE is equivalent to playing with the amplifier ~25ft away from you, which is pretty bad. It is surprising that the best system is actually nrf24, given how old it is (or maybe because of that ?). It also seems to have enough bandwidth to transfer a 24-bit/48kHz signal.

polishdude20 3y ago
With guitar, how would the latency stack up when you add a/d conversion?
- zh3 3y ago
  A/D conversion itself wouldn't add much at the sample level (say 48KHz sample, it's about 50us per sample). However, packetisation will - a 256 byte packet of 128 samples is 128*50us = 6.4ms right there at the transmitter, and the receiver won't notify until the full packet is received. So a naive digital approach would be 12.8ms (2x6.4ms) even before anything else. A pure analogue approach (modulated RF) on the other hand shouldn't have any human-detectable delay - it's effectively distance/speed of light with a bit of a phase shift (addtional delay) introduced by the electronics - should be only a handful of microseconds in total.
  - polishdude20 3y ago
    Analogue would be nice in that regard but wouldn't it be pretty bad with interference, signal quality over distances etc?
    - zh3 3y ago
      It worked that like for many, many years (radio mics) before digital came along. Think of FM radio - if you have a good signal, it's pretty resilient to interference and in a controlled short range environment it is extremely reliable.

nsasch 3y ago

Thanks for sharing your research! I’ve done some similar, but not as thorough, tests with 2.4ghz WiFi and 915MHz LoRa before. My goal is to sync time across multiple devices that go in and out of range of each other to play 60fps light animations and sound within half a frame of each other (8ms), with spare time for some computation. I’ve been surprised at how bad some oscillators (or voltage regulators) can be and their effect on consistent latency. I was having fun experimenting, but this will be really useful to get me back to actually implementing my project.

nanomonkey 3y ago

This is a great breakdown, although it doesn't go into any detail on the distance trade offs of each transmission type, which is an important part of the comparison. I look forward to when they go into this part of the analysis.

noman-land 3y ago

Amazing article and research. Most definitely bookmarked.