4 ms·

I don't quite understand the point that latency doesn't matter much outside of niche applications. If you're using TCP, bandwidth depends on latency, and surel

by svara 5y ago

I don't quite understand the point that latency doesn't matter much outside of niche applications. If you're using TCP, bandwidth depends on latency, and surely no one thinks bandwidth isn't important? This totally does matter in the hundreds-of-milliseconds range, you can use one of the online calculators to confirm this. The other thing is that live video/audio isn't niche at all. Cloudflare could prioritize latency-sensitive traffic like that within its network, and deliver it as close to the users as possible on their respective ends. Am I missing something?

seangrogg 5y ago
While you're correct to think that latency does indeed matter, I think what the article is trying to point out is that very few services are latency sensitive beyond what is traditionally expected. In many applications, there are certain expectations of performance around certain actions, sure. That said, if things go long the user may get annoyed but they won't be deterred. But in some cases, having anything but the highest regard for latency is not only a deterrence, it's basically completely unacceptable to the user. Sure, you want your blog to load fast. But if your blog takes 300ms or 900ms very few people are going to notice the difference despite the 3x disparity. You're unlikely to get a nastygram on Twitter about your blog being unusable. Hell, even websites that take greater-than-one seconds to load are routinely not taken to task by the average user because, quite frankly, while fast is nice it's not a need. But apply that to a competitive, real-time game and if your service takes 70ms RTT for one of the roughly several hundreds of thousands of commands issued to it over the course of a 20 minute span you're performing about average. At 250ms - dramatically less than the kind of latency bound available to a blog post - you're rapidly approaching "completely unplayable" and going to get nasty responses.
- donmcronald 5y ago
  I think a lot of webapps undervalue low latency. I remember when Microsoft was talking about Surface research once and said that most people could start to notice latency >20ms. I think talking about the modern web in milliseconds doesn’t represent the way it really is. Based on my experience everything is measured in seconds and UIs are awful compared to what we had 10-20 years ago. It’s like the old green screens that adept users could outperform, but with better graphics and no buffer/command queue.
  - deleted 5y ago
    [deleted]
  - snowwrestler 5y ago
    Sure, but the point is that the physical light speed delay is a tiny fraction of the typical latency problems on the Internet. It’s a rare web application that is so well-optimized that shrinking the physical wire will make a significant improvement.
johncolanduoni 5y ago
If your only issue is bandwidth delay product, why would you go through all the effort of moving servers closer to the users instead of just increasing TCP transmit buffer size? You don’t even need control over the other side of the connection.
- svara 5y ago
  How large to you want the TCP buffer to get? Hundred-of-mbit/s to GBit/s bandwidths are normal even for consumers now, and so are delays in the hundreds of milliseconds.
  - johncolanduoni 5y ago
    How large does a TCP buffer have to be before the cost of running beefier hardware compares with deploying to tons of edge locations instead of a couple of locations on each continent? At 300ms of latency you only need 37 megabytes per 1Gbps of link capacity. I can’t imagine putting a 100Gbps link on a machine that couldn’t spare 3.7 GB of RAM without breaking a sweat. TCP window scaling (and SACK) exists for a reason.
    - svara 5y ago
      That's a good point about window scaling solving the problem in theory. Still, empirically, latency has a huge impact on throughput. Part of the reason might be that many transfers are actually too small to take full advantage of window scaling? Maybe there are other factors at play as well that I'm not aware of. Check out for example this speed measurement tool from AWS [0], which shows a really drastic influence of physical distance on transfer speeds (particularly for the small files where I get >10x differences, but even at 5 MB I see >3x differences between a close-by zone and those roughly on the other side of the planet). That happens even with their "global accelerator" service which gives them a lot of control over the transfer parameters. [0] https://speedtest.globalaccelerator.aws/#/ https://speedtest.globalaccelerator.aws/#/

by svara 5y ago

seangrogg 5y ago

While you're correct to think that latency does indeed matter, I think what the article is trying to point out is that very few services are latency sensitive beyond what is traditionally expected. In many applications, there are certain expectations of performance around certain actions, sure. That said, if things go long the user may get annoyed but they won't be deterred. But in some cases, having anything but the highest regard for latency is not only a deterrence, it's basically completely unacceptable to the user. Sure, you want your blog to load fast. But if your blog takes 300ms or 900ms very few people are going to notice the difference despite the 3x disparity. You're unlikely to get a nastygram on Twitter about your blog being unusable. Hell, even websites that take greater-than-one seconds to load are routinely not taken to task by the average user because, quite frankly, while fast is nice it's not a need. But apply that to a competitive, real-time game and if your service takes 70ms RTT for one of the roughly several hundreds of thousands of commands issued to it over the course of a 20 minute span you're performing about average. At 250ms - dramatically less than the kind of latency bound available to a blog post - you're rapidly approaching "completely unplayable" and going to get nasty responses.

donmcronald 5y ago
I think a lot of webapps undervalue low latency. I remember when Microsoft was talking about Surface research once and said that most people could start to notice latency >20ms. I think talking about the modern web in milliseconds doesn’t represent the way it really is. Based on my experience everything is measured in seconds and UIs are awful compared to what we had 10-20 years ago. It’s like the old green screens that adept users could outperform, but with better graphics and no buffer/command queue.
- deleted 5y ago
  [deleted]
- snowwrestler 5y ago
  Sure, but the point is that the physical light speed delay is a tiny fraction of the typical latency problems on the Internet. It’s a rare web application that is so well-optimized that shrinking the physical wire will make a significant improvement.

johncolanduoni 5y ago

If your only issue is bandwidth delay product, why would you go through all the effort of moving servers closer to the users instead of just increasing TCP transmit buffer size? You don’t even need control over the other side of the connection.

svara 5y ago
How large to you want the TCP buffer to get? Hundred-of-mbit/s to GBit/s bandwidths are normal even for consumers now, and so are delays in the hundreds of milliseconds.
- johncolanduoni 5y ago
  How large does a TCP buffer have to be before the cost of running beefier hardware compares with deploying to tons of edge locations instead of a couple of locations on each continent? At 300ms of latency you only need 37 megabytes per 1Gbps of link capacity. I can’t imagine putting a 100Gbps link on a machine that couldn’t spare 3.7 GB of RAM without breaking a sweat. TCP window scaling (and SACK) exists for a reason.
  - svara 5y ago
    That's a good point about window scaling solving the problem in theory. Still, empirically, latency has a huge impact on throughput. Part of the reason might be that many transfers are actually too small to take full advantage of window scaling? Maybe there are other factors at play as well that I'm not aware of. Check out for example this speed measurement tool from AWS [0], which shows a really drastic influence of physical distance on transfer speeds (particularly for the small files where I get >10x differences, but even at 5 MB I see >3x differences between a close-by zone and those roughly on the other side of the planet). That happens even with their "global accelerator" service which gives them a lot of control over the transfer parameters. [0] https://speedtest.globalaccelerator.aws/#/ https://speedtest.globalaccelerator.aws/#/