5 ms·
End-to-end congestion control cannot avoid latency spikes (2022)
- westurner 2y ago- "MOOC: Reducing Internet Latency: Why and How" (2023) https://news.ycombinator.com/item?id=37285586#37285733 https://news.ycombinator.com/item?id=37285586#37285733 ; sqm-autorate, netperf, iperf, flent, dslreports_8dn Bufferbloat > Solutions and mitigations: https://en.wikipedia.org/wiki/Bufferbloat#Solutions_and_mitigations https://en.wikipedia.org/wiki/Bufferbloat#Solutions_and_miti...
- mrtesthah 2y agomacOS sonoma supports L4S for this purpose: https://www.theverge.com/23655762/l4s-internet-apple-comcast-latency-speed-bandwidth https://www.theverge.com/23655762/l4s-internet-apple-comcast...
- fmbb 2y agoWhat do you mean “this purpose”? The text linked in OP says > Congestion signalling methods cannot work around this problem either, so our analysis is also valid for Explicit Congestion Notification methods such as Low Latency Low Loss Scalable Throughput (L4S).
- ajb 2y agoL4S (or at least, the dualq implementation) uses a shallow buffer. You can't get 1000ms of latency if your buffer is only 15ms. Although the title claims there's no way round latency spikes, their paper [1] explores the case where latency spikes are traded off against allowing packet loss, which is what you're going to get if you fill a shallow buffer. For real-time traffic, packet dropping is usually preferable. [1] https://arxiv.org/pdf/2111.00488 https://arxiv.org/pdf/2111.00488
- sulandor 2y agothough it supports fair-queuing since 2018
- jchanimal 2y agoIt's fun to think about the theoretical maximum in a multi-link scenario. One thing that pops out of the analysis -- there are diminishing returns to capacity from adding more links. Which means at some point an additional link can begin to reduce capacity as it starts to eat more into maintenance and uptime budget than it offers in capacity.
- 1992spacemovie 2y agoThere are multiple SD-WAN vendors with active-active multipath functionality. Typically the number of parallel active paths is capped at something like 2 or 4. A few esoteric vendors do high numbers (12-16). Fundamentally your premise is correct, but the overhead % is a single digit as I understand it. Slightly different than Amdahl's law in my eyes (transmission of data v computation).
- flapjack 2y agoOne of the solutions they mention is underutilizing links. This is probably a good time to mention my thesis work, where we showed that streaming video traffic (which is the majority of the traffic on the internet) can pretty readily underutilize links on the internet today, without a downside to video QoE! https://sammy.brucespang.com https://sammy.brucespang.com
- clbrmbr 2y agoCan you comment on latency-sensitive video (Meet, Zoom) versus latency-insensitive video (YouTube, Netflix)? Is only the latter “streaming video traffic”?
- flapjack 2y agoWe looked at latency-insensitive like YouTube and Netflix (which were a bit more than 50% of internet traffic last year [1]). I'd bet you could do something similar with Meet and Zoom–my understanding is video bitrates for those services are lower than for e.g. Netflix which we showed are much lower than network capacities. But it might be tricky because of the latency-sensitivity angle, and we did not look into it in our paper. [1] https://www.sandvine.com/hubfs/Sandvine_Redesign_2019/Downloads/2024/GIPR/GIPR%202024.pdf https://www.sandvine.com/hubfs/Sandvine_Redesign_2019/Downlo...
- mikepavone 2y ago> Meet and Zoom–my understanding is video bitrates for those services are lower than for e.g. Netflix For a given quality, bitrate will generally be higher in RTC apps (though quality may be lower in general depending on the context and network conditions obviously) because of tradeoffs between encoding latency and efficiency. However, RTC apps generally already try to underutilize links because queuing is bad for latency and latency matters a lot for the RTC case.
- sulandor 2y agothe term “streaming video" usually refers to the fact that the data is sent slower than the link capacity (but intermittently faster than the content bitrate) op used the term presumably to describe "live content" eg. the source material is not available as a whole (because the recording is not finished); which can be considered a subset of "streaming video" the sensitivity in regard to transport characteristics stems from the fact that "live content" places an upper bound for the time required for processing and transferring the content-bits to the clients (for it to be considered "live").
- bobmcnamara 2y agoThis result is explainable by Danish tandem q theory.
- fmbb 2y agoCan someone explain this to a layman? Because it seems to me the four solutions proposed are: 1. Seeing the future 2. Building a ten times higher capacity network 3. Breaking Net neutrality by deprioritizing traffic that someone deems “not latency sensitive” 4. Flood the network with more redundant packets Or is the whole text a joke going over my head?
- DowsingSpoon 2y agoHi! I’m also a layman who doesn’t really know what he’s talking about. The article ends with “ I will leave you with a question: Are we trying to make end-to-end congestion control work for cases where it can’t possibly work?” So, it seems to me that there may not be any good solutions to latency spikes. The article is basically pointing out that you either pursue one of the unfortunate solutions mentioned, or be resigned to accept that no congestion control mechanism will ever be sufficient to eliminate the spikes. This seems a valuable message to the people who might be involved in developing the sort congestion control mechanisms they’re talking about.
- fmbb 2y agoHaha joke’s on me I guess, not reading the _headline_ and skimming the intro so not getting the point.
- Sohcahtoa82 2y agoI think the author is saying that there ARE solutions, but none of them are really viable. Seeing the future obviously can't happen. Building a higher capacity network is just wasted money. Breaking NN is going to be unpopular, not to mention determination of "not latency sensitive" is going to be difficult to impossible unless there's a "not latency sensitive" flag on TCP packets that people actually use in good faith. And flooding the network with more redundant packets is just going to be a colossal waste of bandwidth and could easily make congestion issues worse.
- travisb 2y agoDSCP already exists, but there's never been an Internet-wide incentive to use it correctly. It is theoretically plausible that end-users could mark their packets 'latency/jitter sensitive' or 'high throughput'. If there was an Internet-wide consensus that those two options made useful trade-offs then there would be an incentive to use it correctly. That would be as NN-safe as any other scheme. For example, maybe 'latency/jitter sensitive' buffers less, but at a 25+% throughput penalty. 'High throughput' would then be faster, but have much higher latency/jitter. The trick would be ensuring fairness without requiring too much hardware at scale.
- comex 2y agoSounds like a good argument for using a CDN. Or to phrase it more generally, an intermediary that is as close as possible to the host experiencing the fluctuating bandwidth bottleneck, while still being on the other side of the bottleneck. That way it can detect bandwidth drops quickly and handle them more intelligently than by dropping packets - for instance, by switching to a lower quality video stream (or even re-encoding on the fly).
- scottlamb 2y ago> That is not true for networks where link capacity can change rapidly, such as Wi-Fi and 5G. Is this problem almost exclusively to do with the "last-mile" leg of the connection to the user? (Or the two legs, in the case of peer-to-peer video chats.) I would expect any content provider or Internet backbone connection to be much more stable (and generally over-provisioned too). In particular, there may be occasional routing changes but a given link should be a fiber line with fixed total capacity. Or is having changes in what other users are contending for that capacity effectively the same problem?
- lo0dot0 2y agoThe entire network is usually oversubscribed so there can be congestion. It is usually not over-provisioned because that would be too expensive.
- scottlamb 2y agoI don't agree. When I think about my experience at say Google, there are quite a variety of ways we'd avoid graphs hitting 100% (or say 90% if that's where the user experience takes a marked turn for the worse). * We absolutely would spend the money to avoid globally hitting that number. * We'd use ToS so that user-facing TCP traffic wouldn't get dropped, but instead some less latency-sensitive and more loss-resistant transfer protocol traffic would instead. * We'd have several levels/types of load balancing from DNS-based as they come into the network (typically we'd direct to the closest relevant datacenter but less so as it gets overloaded) to route advertising to Maglev<->GFE balancing to GFE<->application balancing and so on. * etc. I would expect that'd be true to some extent for any content provider. There are surely some problematic hops in the network (I've seen alleged leaked graphs of Comcast backbone traffic flattening out at 100% at the same time every day) but the entire network is oversubscribed...and running out of capacity regularly in practice? No way.
- Hikikomori 2y agoIts oversubscribed as many users share the same link at some point and that link is not big enough to allow all users to use all their bandwidth at the same time. ISPs will oversubscribe and add more capacity when needed to avoid congestion, around 70-95% utilization depending on link size. American ISPs seems to not care as much though.
- dilyevsky 2y agoThere's a paper by Google where they demonstrated that you can successfully push utilization far beyond of what is suggested here[0] [0] - https://www.cs.princeton.edu/courses/archive/fall17/cos561/papers/espresso17.pdf https://www.cs.princeton.edu/courses/archive/fall17/cos561/p...
- kccqzy 2y agoSection 6.2 of the paper says Google does traffic engineering (i.e. designating some traffic to be latency tolerant). That's just what solution 3 is in the article. For a single company, of course it can prioritize different traffic differently. It would be more troublesome for an ISP to do the same.
- sambazi 2y agolooks like they only care for drops and not latency
- dilyevsky 2y agoDrops is what happens when the queue is filling up (via AQM) but they track queue utilization as well