3 ms·

Over the weekend I setup one of the new consumer mesh products that's available, the Linksys Velop, with 9 nodes covering a good sized area between two homes.

by orware 9y ago

Over the weekend I setup one of the new consumer mesh products that's available, the Linksys Velop, with 9 nodes covering a good sized area between two homes. One thing I've been noticing though is that there is considerable latency/packet loss at the moment (there is only one wired backhaul at the primary node and all of the other nodes are connected to each other wirelessly). I've been running Ping Plotter against to all of the nodes and there seems to be considerable packet loss (a few percent) and spikes in latency (average for the two closest nodes to my laptop is about 15 ms, the middle ones out a ways are about 30-40 ms, and the furthest ones are at about 60 ms) but the spikes can be in the hundreds or even thousands of ms. The area covered is about a 500 ft by 120 ft rectangle more or less (with my house on the bottom left of that rectangle and the other home on the bottom right of that rectangle). My question would be...would this BBR algorithm help in some way to reduce the latency/packet loss in a situation like this? Or does it only apply for these other situations that Google would normally be encountering/dealing with? Thanks for the input!

mangix 9y ago
BBR solves a different problem. The problem you have is wifi being terrible. There's massive buffering in the firmware which you can't get rid of.
- orware 9y ago
  In this case do you mean the firmware being provided by Linksys for the hardware, or an additional layer of firmware embedded into the wireless hardware in some way? It actually looks like for the Velop they are using some form of OpenWRT it looks like from what I can see when I pull up the sysinfo page in the router so it makes it seem like they would have the ability to customize/tweak the buffering settings in some way (I took Georgia Tech's Networking class about two years ago now, but it was pretty neat to learn about the buffer bloat problem there in that course and how they were mentioning bigger buffers weren't necessarily better for performance).
  - wtallis 9y ago
    The radio's firmware, not the Linux OS running on the application processor. All 802.11ac radios have closed-source firmware even when there are open-source Linux drivers to communicate with the NIC. The 802.11n chipsets by Atheros didn't use proprietary firmware and exposed a fairly low level interface to the host system. This led to the open-source ath9k Linux driver being the platform of choice for people trying to fix WiFi in general or improve the Linux WiFi stack in specific.
- adrianmonk 9y ago
  That doesn't seem like all that different of a problem. I'm certainly not an expert on BBR, but from reading the description, the design goals seem to explicitly include dealing with buffers better (by making efforts to not fill them to the max) and being less skittish about packet loss. Specifically, the description (in the git commit) says it has a "congestion control algorithm that reacts to actual congestion, not packet loss or transient queue delay" and that it estimates the size of the queue it probably created and paces packets in order to "utilize the pipe without creating excess queue". (And that last part addresses large buffers. Even if they can't be turned off, they only fill up if you send enough data to fill them.) Obviously it isn't magic and there is only so much any algorithm can do in the face of a cruddy physical network layer, but the traditional algorithm makes a bad situation much worse than it has to be, so there is still the potential for a newer algorithm (like BBR) to make a big improvement. Anyway, more to the point, cruddy wifi is what a lot of people use to browse the web, so it's not surprising to me if Google tried to account for that in their design.
deleted 9y ago
[deleted]
sbierwagen 9y ago
Sounds like just bad physical-layer connectivity, nothing to do with TCP. Most of my experience with wifi mesh is from years ago, with pure 2.4ghz stuff, back when it basically didn't work at all. How close are the nodes? Are there any long multi-hop chains? (Repeater talking to repeater talking to root. The more hops, the worse it works)

by orware 9y ago

mangix 9y ago

BBR solves a different problem. The problem you have is wifi being terrible. There's massive buffering in the firmware which you can't get rid of.

orware 9y ago
In this case do you mean the firmware being provided by Linksys for the hardware, or an additional layer of firmware embedded into the wireless hardware in some way? It actually looks like for the Velop they are using some form of OpenWRT it looks like from what I can see when I pull up the sysinfo page in the router so it makes it seem like they would have the ability to customize/tweak the buffering settings in some way (I took Georgia Tech's Networking class about two years ago now, but it was pretty neat to learn about the buffer bloat problem there in that course and how they were mentioning bigger buffers weren't necessarily better for performance).
- wtallis 9y ago
  The radio's firmware, not the Linux OS running on the application processor. All 802.11ac radios have closed-source firmware even when there are open-source Linux drivers to communicate with the NIC. The 802.11n chipsets by Atheros didn't use proprietary firmware and exposed a fairly low level interface to the host system. This led to the open-source ath9k Linux driver being the platform of choice for people trying to fix WiFi in general or improve the Linux WiFi stack in specific.
adrianmonk 9y ago
That doesn't seem like all that different of a problem. I'm certainly not an expert on BBR, but from reading the description, the design goals seem to explicitly include dealing with buffers better (by making efforts to not fill them to the max) and being less skittish about packet loss. Specifically, the description (in the git commit) says it has a "congestion control algorithm that reacts to actual congestion, not packet loss or transient queue delay" and that it estimates the size of the queue it probably created and paces packets in order to "utilize the pipe without creating excess queue". (And that last part addresses large buffers. Even if they can't be turned off, they only fill up if you send enough data to fill them.) Obviously it isn't magic and there is only so much any algorithm can do in the face of a cruddy physical network layer, but the traditional algorithm makes a bad situation much worse than it has to be, so there is still the potential for a newer algorithm (like BBR) to make a big improvement. Anyway, more to the point, cruddy wifi is what a lot of people use to browse the web, so it's not surprising to me if Google tried to account for that in their design.

deleted 9y ago

[deleted]

sbierwagen 9y ago

Sounds like just bad physical-layer connectivity, nothing to do with TCP. Most of my experience with wifi mesh is from years ago, with pure 2.4ghz stuff, back when it basically didn't work at all. How close are the nodes? Are there any long multi-hop chains? (Repeater talking to repeater talking to root. The more hops, the worse it works)