6 ms·

I have spent hours arguing with someone at my work that the issue we are experiencing at our remote locations is not due to bandwidth, but latency. These graphi

I have spent hours arguing with someone at my work that the issue we are experiencing at our remote locations is not due to bandwidth, but latency. These graphics are exactly what I've been looking for to help get my point across. People do a speedtest and see low (sub-100) numbers and think that's why their video call is failing. Never mind the fact that Zoom only needs 3 Mbps for 1080p video.

dtaht 2y ago
BITAG published this a while back. https://www.bitag.org/latency-explained.php https://www.bitag.org/latency-explained.php It's worth a read.
dtaht 2y ago
speedtest.net added support for tracking latency under load a few years ago. they show ping during up/dl now. That's the number to show your colleague. However they tend to use something like the 75th percentile and throw out real data. The waveform bufferbloat test does 95% and supplies whisker charts. cloudflare also. No web test tests up + down at the same time, which is the worst scenario. crusader and flent.org's rrul test do. Rathan than argue with your colleague, why not just slap an OpenWrt box as a transparent bridge inline and configure CAKE SQM?
- thomasjudge 2y ago
  Would you put the "OpenWrt box as a transparent bridge inline" between your home router and the cable modem, or on the house side of the home router?
  - dtaht 2y ago
    I would replace the home router with an OpenWrt router.
    - matheusmoreira 2y ago
      One of the best things I've ever done. OpenWrt is so good. SQM helps a lot with latency.
- imp0cat 2y ago
  Next time you need to assess your connection's capabilities, try https://speed.cloudflare.com/ https://speed.cloudflare.com/ instead of speedtest.net. Much more informative.
EvanAnderson 2y ago
Latency is a cruel mistress. Had a Customer who was using an old Win32 app that did a ton of individual SELECT queries against the database server to render the UI. They tried to put it on the end of a VPN connection and it was excruciating. The old vendor kept trying to convince them to add bandwidth. Fortunately the Customer asked the question "Why does the app work fine across the 1.544Mbps T1 to our other office?" (The T1 had sub-5ms latency.)
- chrismorgan 2y ago
  I was involved in some physical network inventory software a dozen years ago. One team produced some new web-based software for it, for use by field agents and such. The first version we got to review was rather bad in various important areas; my favourite was that search would take over thirty seconds in common real-world deployment environment: it was implemented in .NET stuff that makes server calls easy to do by accident and in unnecessarily blocking fashion, and searched by each of the 34 entity types individually, in sequence; and some .NET session thing meant that if the connection had been idle for ten minutes or something, it would even then need to retry every request that got queued while the session was “stale”, which was all of them. So you ended up with 68 sequential requests, on up to half a second’s latency (high-latency 3G or 2G or geostationary satellite)… so yeah, 30 seconds. They’d only developed it with sub-millisecond latency to the server, so they never noticed this. I don’t think it was a coincidence that the team was US-based: in Australia, we’re used to internet stuff having hundreds of milliseconds of latency, since so much of the internet is US-hosted, so I think Australians would be more likely to notice such issues early on. All those studies about people abandoning pages if they take more than two seconds to load… at those times, it was a rare page that has even started rendering that soon, because of request waterfalls and high latency. (These days, it’s somewhat more common to have CDNs fix the worst of the problem.)
  - dietr1ch 2y ago
    I think that latency people grow up with has a huge impact on how careful they are when using the internet. Having gotten my hands on an experimental 128kbps link early on, but later and moving to the countryside with a 56kb-1Mbps really spotty connection made me really appreciate local state as every time things blocked on the internet made it pretty notorious. I'm glad there's a push for synchronized, local-first state now, as roaming around on mobile or with a laptop hopping on wifi can only perform nicely with local state.
danpalmer 2y ago
Funnily enough I have found since moving from London to Sydney that people here are far more understanding of the difference between latency and throughput. Being 200ms from anyone else on the internet will do that to you!
buginprod 2y ago
I dont see how people miss latency. It is the only other number shown on the speed check screen! No curiousity as to why that is there? I mean I bet they do care about litres/100km for their car AND 0-100km accelerarion (and many other stats)
izacus 2y ago
Not just latency, but also jitter. Jitter was the biggest issue we had when broadcasting and streaming video. You don't need a lot of bandwidth, latency can be surivived... but jitter will ruin your experience like nothing else.
Roark66 2y ago
That is why I've been successfully working from home for almost a decade starting on an LTE connection that was 5Mb up and 10Mb down(notice this is small b as in bits). No problem at all... Why because most of the time latency was good. I'm still on the same LTE connection, but everyone kept telling me how my speeds are crap and how I should update to a new LTE cat 21 router. So I got one of more popular models ZTE MF289F. And the speed increased to 50Mb up 75Mb down on a speed test. But all my calls suddenly felt very choppy and the perceived Web browsing was unbearably slow... What happened? Well, the router would just decide every day or so to up it's latency to Google.com from 15ms to 150ms until it was restarted. But that is not all. Even when the ping latency was fine it still felt slower than my ancient tplink lte router... So the zte went into a drawer waiting for the times I'll have time to put Linux on it. And the tplink went back on top of my antenna mast.
- dtaht 2y ago
  In a lot of cases older is better. See also https://github.com/lynxthecat/cake-autorate https://github.com/lynxthecat/cake-autorate for an active measurement tool...
guappa 2y ago
"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway." -- Andrew S. Tanenbaum

dtaht 2y ago

BITAG published this a while back. https://www.bitag.org/latency-explained.php https://www.bitag.org/latency-explained.php It's worth a read.

dtaht 2y ago

speedtest.net added support for tracking latency under load a few years ago. they show ping during up/dl now. That's the number to show your colleague. However they tend to use something like the 75th percentile and throw out real data. The waveform bufferbloat test does 95% and supplies whisker charts. cloudflare also. No web test tests up + down at the same time, which is the worst scenario. crusader and flent.org's rrul test do. Rathan than argue with your colleague, why not just slap an OpenWrt box as a transparent bridge inline and configure CAKE SQM?

thomasjudge 2y ago
Would you put the "OpenWrt box as a transparent bridge inline" between your home router and the cable modem, or on the house side of the home router?
- dtaht 2y ago
  I would replace the home router with an OpenWrt router.
  - matheusmoreira 2y ago
    One of the best things I've ever done. OpenWrt is so good. SQM helps a lot with latency.
imp0cat 2y ago
Next time you need to assess your connection's capabilities, try https://speed.cloudflare.com/ https://speed.cloudflare.com/ instead of speedtest.net. Much more informative.

EvanAnderson 2y ago

Latency is a cruel mistress. Had a Customer who was using an old Win32 app that did a ton of individual SELECT queries against the database server to render the UI. They tried to put it on the end of a VPN connection and it was excruciating. The old vendor kept trying to convince them to add bandwidth. Fortunately the Customer asked the question "Why does the app work fine across the 1.544Mbps T1 to our other office?" (The T1 had sub-5ms latency.)

chrismorgan 2y ago
I was involved in some physical network inventory software a dozen years ago. One team produced some new web-based software for it, for use by field agents and such. The first version we got to review was rather bad in various important areas; my favourite was that search would take over thirty seconds in common real-world deployment environment: it was implemented in .NET stuff that makes server calls easy to do by accident and in unnecessarily blocking fashion, and searched by each of the 34 entity types individually, in sequence; and some .NET session thing meant that if the connection had been idle for ten minutes or something, it would even then need to retry every request that got queued while the session was “stale”, which was all of them. So you ended up with 68 sequential requests, on up to half a second’s latency (high-latency 3G or 2G or geostationary satellite)… so yeah, 30 seconds. They’d only developed it with sub-millisecond latency to the server, so they never noticed this. I don’t think it was a coincidence that the team was US-based: in Australia, we’re used to internet stuff having hundreds of milliseconds of latency, since so much of the internet is US-hosted, so I think Australians would be more likely to notice such issues early on. All those studies about people abandoning pages if they take more than two seconds to load… at those times, it was a rare page that has even started rendering that soon, because of request waterfalls and high latency. (These days, it’s somewhat more common to have CDNs fix the worst of the problem.)
- dietr1ch 2y ago
  I think that latency people grow up with has a huge impact on how careful they are when using the internet. Having gotten my hands on an experimental 128kbps link early on, but later and moving to the countryside with a 56kb-1Mbps really spotty connection made me really appreciate local state as every time things blocked on the internet made it pretty notorious. I'm glad there's a push for synchronized, local-first state now, as roaming around on mobile or with a laptop hopping on wifi can only perform nicely with local state.

danpalmer 2y ago

Funnily enough I have found since moving from London to Sydney that people here are far more understanding of the difference between latency and throughput. Being 200ms from anyone else on the internet will do that to you!

buginprod 2y ago

I dont see how people miss latency. It is the only other number shown on the speed check screen! No curiousity as to why that is there? I mean I bet they do care about litres/100km for their car AND 0-100km accelerarion (and many other stats)

izacus 2y ago

Not just latency, but also jitter. Jitter was the biggest issue we had when broadcasting and streaming video. You don't need a lot of bandwidth, latency can be surivived... but jitter will ruin your experience like nothing else.

Roark66 2y ago

That is why I've been successfully working from home for almost a decade starting on an LTE connection that was 5Mb up and 10Mb down(notice this is small b as in bits). No problem at all... Why because most of the time latency was good. I'm still on the same LTE connection, but everyone kept telling me how my speeds are crap and how I should update to a new LTE cat 21 router. So I got one of more popular models ZTE MF289F. And the speed increased to 50Mb up 75Mb down on a speed test. But all my calls suddenly felt very choppy and the perceived Web browsing was unbearably slow... What happened? Well, the router would just decide every day or so to up it's latency to Google.com from 15ms to 150ms until it was restarted. But that is not all. Even when the ping latency was fine it still felt slower than my ancient tplink lte router... So the zte went into a drawer waiting for the times I'll have time to put Linux on it. And the tplink went back on top of my antenna mast.

dtaht 2y ago
In a lot of cases older is better. See also https://github.com/lynxthecat/cake-autorate https://github.com/lynxthecat/cake-autorate for an active measurement tool...

guappa 2y ago

"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway." -- Andrew S. Tanenbaum