33 ms·

QUIC is not quick enough over fast internet

664 points by Shank 2y ago

JoshTriplett 2y ago
Seems to be available on arXiv: https://arxiv.org/pdf/2310.09423 https://arxiv.org/pdf/2310.09423
- Tempest1981 2y ago
  The page headings say "Conference'17, July 2017" -- why is that? Although the sidebar on page 1 shows "13 Oct 2023".
  - mrngm 2y ago
    It's likely the authors used an existing conference template to fit in their paper's contents. Upon sending it to the conference, the editors can easily fit the contents in their prescribed format, and the authors know how many characters they can fit in the page limit. arXiv typically contains pre-prints of papers. These may not have been peer-reviewed, and the contents may not reflect the actual "published" paper that was accepted (and/or corrected after peer review) to a conference or journal. arXiv applies a watermark to the submitted PDF such that different versions are distinguishable on download.
jacob019 2y ago
Maybe moving the connection protocol into userspace isn't such a great plan.
- foota 2y ago
  I don't have access to the article, but they're saying the issue is due to client side ack processing. I suspect they're testing at bandwidths far beyond what's normal for consumer applications.
  - dartharva 2y ago
    It's available on arxiv and nope, they are testing mostly for regular 4G/5G speeds. https://arxiv.org/pdf/2310.09423 https://arxiv.org/pdf/2310.09423
    - DannyBee 2y ago
      4g tops out at 1gbps only when one person is on the network. 5g tops out at ~10gbps (some 20gbps i guess) only when one person is on the network. They are testing at 1gbps. This is not regular 4g speed for sure, and it's a rare 5g speed. regular 5g speed is (in the US) 40-50mbps, so, 20x slower than they are testing.
      - dartharva 2y ago
        Still won't be beyond normal consumer applications' capacity, right?
        DannyBee 2y ago
        correct
      - izend 2y ago
        What about 1gbps fiber at home, it is becoming common in Canada. I have 1gbps up/down.
        DannyBee 2y ago
        This would affect that. As said, i was only replying to the claim that this affects things at 4g/5g cell phone speeds, which it clearly does not, by their own data.
      - vrighter 2y ago
        Gigabit fiber internet is quite cheap and increasingly available (I'm not from the US). I don't just use the internet over a 4/5g connection. This definitely affects more people than you think.
- 01HNNWZ0MV43FF 2y ago
  Does QUIC mandate that, or is that just the stepping stone until the chicken-and-egg problem is solved and we get kernel support?
  - wmf 2y ago
    On mobile the plan is to never use kernel support so that apps can have the latest QUIC on old kernels.
  - vlovich123 2y ago
    As others in the thread summarized the paper as saying the issue is ack offload. That has nothing to do with whether the stack is in kernel space or user space. Indeed there’s some concern about this inevitable scenario because the kernel is so slow moving, updates take much longer to propagate to applications needing them without a middle ground whereas as user space stacks they can update as the endpoint applications need them to.
  - kmeisthax 2y ago
    No, but it depends on how QUIC works, how Ethernet hardware works, and how much you actually want to offload to the NIC. For example, QUIC has TLS encryption built-in, so anything that's encrypted can't be offloaded. And I don't think most people want to hand all their TLS keys to their NIC[0]. At the very least you probably would have to assign QUIC its own transport, rather than using UDP as "we have raw sockets at home". Problem is, only TCP and UDP reliably traverse the Internet[1]. Everything in the middle is sniffing traffic, messing with options, etc. In fact, Google rejected an alternate transport protocol called SCTP (which does all the stream multiplexing over a single connection that QUIC does) specifically because, among other things, SCTP's a transport protocol and middleboxes choke on it. [0] I am aware that "SSL accelerators" used to do exactly this, but in modern times we have perfectly good crypto accelerators right in our CPU cores. [1] ICMP sometimes traverses the internet, it's how ping works, but a lot of firewalls blackhole ICMP. Or at least they did before IPv6 made it practically mandatory to forward ICMP packets.
    - _flux 2y ago
- throwaway984393 2y ago
  [dead]
- kccqzy 2y ago
  The flexibility and ease of changing a userspace protocol IMO far outweighs anything else. If the performance problem described in this article (which I don't have access to) is in userspace QUIC code, it can be fixed and deployed very quickly. If similar performance issue were to be found in TCP, expect to wait multiple years.
  - vrighter 2y ago
    Well, the problem is probably that it is in userspace in the first place.
- mrweasel 2y ago
  Maybe moving the entire application to the browser/cloud wasn't the best idea for a large number of use cases? Video streaming, sure, but we're already able to stream 4K video over a 25Mbit line. With modern internet connections being 200Mbit to 1Gbit, I don't see that we need the bandwidth in private homes. Maybe for video conferencing in large companies, but that also doesn't need to be 4K. The underlying internet protocols are old, so there's no harm in assessing if they've outlived their usefulness. However, we should also consider in web applications and "always connected" is truly the best solution for our day to day application needs.
  - kuschku 2y ago
    > With modern internet connections being 200Mbit to 1Gbit, I don't see that we need the bandwidth in private homes Private connections tend to be asymmetrical. In some cases, e.g. old DOCSIS versions, that used to be due to technical necessity. Private connections tend to be unstable, the bandwidth fluctuates quite a bit. Depending on country, the actually guaranteed bandwidth is somewhere between half of what's on the sticker, to nothing at all. Private connections are usually used by families, with multiple people using it at the same time. In recent years, you might have 3+ family members in a video call at the same time. So if you're paying for a 1000/50 line (as is common with DOCSIS deployments), what you're actually getting is usually a 400/20 line that sometimes achieves more. And those 20Mbps upload are now split between multiple people. At the same time, you're absolutely right – Gigabit is enough for most people. Download speeds are enough for quite a while. We should instead be increasing upload speeds and deploying FTTH and IPv6 everywhere to reduce the latency.
    - throwaway2037 2y ago
      This is a great post. I often forget that home Internet connections are frequently shared between many people. This bit: > IPv6 everywhere to reduce the latency I am not an expert on IPv4 vs IPv6. Teach me: How will migrating to IPv6 reduce latency? As I understand, a lot of home Internet connections are always effectively IPv6 via CarrierNAT. (Am I wrong? Or not relevant to your point?)
- simiones 2y ago
  The problem is that the biggest win by far with QUIC is merging encryption and session negotiation into a single packet, and the kernel teams have been adamant about not wanting to maintain encryption libraries in kernel. So, QUIC or any other protocol like it being in kernel is basically a non-starter.
- suprjami 2y ago
  Looking forward to QUIC in the kernel. Linux already has kTLS.
JoshTriplett 2y ago
In the early days of QUIC, many people pointed out that the UDP stack has had far far less optimization put into it than the TCP stack. Sure enough, some of the issues identified here arise because the UDP stack isn't doing things that it could do but that nobody has been motivated to make it do, such as UDP generic receive offload. Papers like this are very likely to lead to optimizations both obvious and subtle.
- nextaccountic 2y ago
  Do you mean that under the same workload, TCP will perform better?
- Animats 2y ago
  What is UDP offload going to do? UDP barely does anything but queue and copy. Linux scheduling from packet-received to thread has control is not real-time, and if the CPUs are busy, may be rather slow. That's probably part of the bottleneck. The embarrassing thing is that QUIC, even in Google's own benchmarks, only improved performance by about 10%. The added complexity probably isn't worth the trouble. However, it gave Google control of more of the stack, which may have been the real motivation.
  - Vecr 2y ago
    I think one of the original drivers was the ability to quickly tweak parameters, after Linux rejected what I think was userspace adjustment of window sizing to be more aggressive than the default. The Linux maintainers didn't want to be responsible for congestion collapse, but UDP lets you spray packets from userspace, so Google went with that.
  - JoshTriplett 2y ago
    Among other things, GRO (receive offloading) means you can get more data off of the network card in fewer operations. Linux has receive packet steering, which can help with getting packets from the network card to the right CPU and the right userspace thread without moving from one CPU's cache to another.
mholt 2y ago
I don't have access to the paper but based on the abstract and a quick scan of the presentation, I can confirm that I have seen results like this in Caddy, which enables HTTP/3 out of the box. HTTP/3 implementations vary widely at the moment, and will likely take another decade to optimize to homogeneity. But even then, QUIC requires a lot of state management that TCP doesn't have to worry about (even in the kernel). There's a ton of processing involved with every UDP packet, and small MTUs, still engrained into many middle boxes and even end-user machines these days, don't make it any better. So, yeah, as I felt about QUIC ... oh, about 6 years ago or so... HTTP/2 is actually really quite good enough for most use cases. The far reaches of the world and those without fast connections will benefit, but the majority of global transmissions will likely be best served with HTTP/2. Intuitively, I consider each HTTP major version an increased order of magnitude in complexity. From 1 to 2 the main complexities are binary (that's debatable, since it's technically simpler from an encoding standpoint), compression, and streams; then with HTTP/3 there's _so, so much_ it does to make it work. It _can_ be faster -- that's proven -- but only when networks are slow. TCP congestion control is its own worst enemy, but when networks aren't congested (and with the right algorithm)... guess what. It's fast! And the in-order packet transmissions (head-of-line blocking) makes endpoint code so much simpler and faster. It's no wonder TCP is faster these days when networks are fast. I think servers should offer HTTP/3 but clients should be choosy when to use it, for the sake of their own experience/performance.
- deleted 2y ago
  [deleted]
- altairprime 2y ago
  The performance gap is shown to be due to hardware offloading, not due to congestion control, in the arxiv paper above.
  - vlovich123 2y ago
    And because Quic is encrypted at a fundamental level, offload likely means needing to share keys with the network card which is a trust concern.
deleted 2y ago
[deleted]
sbstp 2y ago
Even HTTP/2 seems to have been rushed[1]. Chrome has removed support for server push. Maybe more thought should be put into these protocols instead of just rebranding whatever Google is trying to impose on us. [1] https://varnish-cache.org/docs/trunk/phk/h2againagainagain.html https://varnish-cache.org/docs/trunk/phk/h2againagainagain.h...
- surajrmal 2y ago
  It's okay to make mistakes, that's how you learn and improve. Being conservative has drawbacks of its own. Id argue we need more parties involved earlier in the process rather than just time.
  - zdragnar 2y ago
    It's a weird balancing act. On the other hand, waiting for everyone to agree on everything means that the spec will take a decade or two for everyone to come together, and then all the additional time for everyone to actively support it. AJAX is a decent example. Microsoft's Outlook Web Access team implemented XMLHTTP as an activex thing for IE 5 and soon the rest of the vendors adopted it as a standard thing as XmlHttpRequest objects. In fact, I suspect the list of things that exist in browsers because one vendor thought it was a good idea and everyone hopped on board is far, far longer than those designed by committee. Often times, the initially released version is not exactly the same that everyone standardized on, but they all get to build on the real-world consequences of it. I happen to like the TC39 process https://tc39.es/process-document/ https://tc39.es/process-document/ which requires two live implementations with use in the wild for something to get into the final stage and become an official part of the specification. It is obviously harder for something like a network stack than a JavaScript engine to get real world use and feedback, but it has helped to keep a lot of the crazier vendor specific features at bay.
  - arcbyte 2y ago
    It's okay to make mistakes, but its not okay to ignore the broad consensus that HTTP2 was TERRIBLY designed and then admit it 10 years later as if it was unknowable. We knew it was bad.
botanical 2y ago
> we identify the root cause to be high receiver-side processing overhead I find this to be the issue when it comes to Google, and I bet it was known before hand; pushing processing to the user. For example, the AV1 video codec was deployed when no consumer had HW decoding capabilities. It saved them on space at the expense of increased CPU usage for the end-user. I don't know what the motive was there; it would still show that they are carbon-neutral while billions are busy processing the data.
- anfilt 2y ago
  Well I will say if your running servers hit billions of times per day. Offloading processing to the client when safe to do so starts make sense financially. Google does not have to pay for your CPU or storage usage ect... Also I will say if said overhead is not too much it's not that bad of a thing.
- kccqzy 2y ago
  This is indeed an issue but it's widespread and everyone does it, including Google. Things like servers no longer generating actual dynamic HTML, replaced with servers simply serving pure data like JSON and expecting the client to render it into the DOM. It's not just Google that doesn't care, but the majority of web developers also don't care.
  - SquareWheel 2y ago
    There's clearly advantages to writing a web app as an SPA, otherwise web devs wouldn't do it. The idea that web devs "don't care" (about what exactly?) really doesn't make any sense. Moving interactions to JSON in many cases is just a better experience. If you click a Like button on Facebook, which is the better outcome: To see a little animation where the button updates, or for the page to reload with a flash of white, throw away the comment you were part-way through writing, and then scroll you back to the top of the page? There's a reason XMLHttpRequest took the world by storm. More than that, jQuery is still used on more than 80% of websites due in large part to its legacy of making this process easier and cross-browser.
Sparkyte 2y ago
Maybe I'm the only person who thinks that trying to make existing internet protocols faster is wasted energy. But who am I to say anything.
- cheema33 2y ago
  > Maybe I'm the only person who thinks that trying to make existing internet protocols faster is wasted energy. But who am I to say anything. If you have a valid argument to support your claim, why not present it?
  - Sparkyte 2y ago
    They are already expected standards so when you create optimizations you're building on functions that need to be supported additionally on top of them. This leads to incompatibility and sometimes often worse performance as what is being experienced here with QUIC. You can read more about such things from, The Evolution of the Internet Congestion Control. https://groups.csail.mit.edu/ana/Publications/The_Evolution_of_Internet_Congestion_Bauer_Clark_Lehr_TPRC_2009.pdf https://groups.csail.mit.edu/ana/Publications/The_Evolution_... A good solution is to create a newer protocol when the limits of an existing protcol are exceeded. No one thought of needing HTTPS long ago and now we have 443 for HTTP security. If we need something to be faster and it has already achieved an arbitrary limit for the sake of backward compatibility it would be better to introduce a new protocol. I dislike the idea that we're turning into another Reddit where we are pointing fingers at people for updoots. If you dislike my opinion please present one equal to where that can be challenged.
    - likis 2y ago
      You posted your opinion without any kind of accompanying argument, and it was also quite unclear what you meant. Whining about being a target and being downvoted is not really going to help your case. I initially understood your first post as: "Let's not try to make the internet faster" With this reply, you are clarifying your initial post that was very unclear. Now I understand it as: "Let's not try to make existing protocols faster, let's make new protocols instead"
apitman 2y ago
Currently chewing my way laboriously through RFC9000. Definitely concerned by how complex it is. The high level ideas of QUIC seem fairly straight forward, but the spec feels full of edge cases you must account for. Maybe there's no other way, but it makes me uncomfortable. I don't mind too much as long as they never try to take HTTP/1.1 from me.
- ironmagma 2y ago
  Considering they can’t really even make IPv6 happen, that seems like a likely scenario.
  - BartjeD 2y ago
    https://www.google.com/intl/en/ipv6/statistics.html https://www.google.com/intl/en/ipv6/statistics.html I think it's just your little corner of the woods that isn't adopting it. Over here the trend is very clearly to move away from IPv4, except for legacy reasons.
    - apitman 2y ago
      The important milestone is when it's safe to turn IPv4 off. And that's not going to happen as long as any country hasn't fully adopted it, and I don't think that's ever going to happen. For better or worse NAT handles outgoing connections and SNI routing handles incoming connections for most use cases. Self-hosting is the most broken but IMO that's better handled with tunneling anyway so you don't expose your home IP.
      - jeroenhd 2y ago
        IPv4 doesn't need to be off. Hacks and workarounds like DS-Lite can stay with us forever, just like hacks and workarounds like NAT and ALGs will.
        consp 2y ago
        DS-lite (aka CGNAT), now we don't need to give the costumers a proper IP address anymore. It should be banned as it limits IPv6 adoption and it getting more and more use for "customers own good" and is annoying as hell to work around.
jauntywundrkind 2y ago
I wonder if these results reproduce on Windows. Is there any TCP offload or GSO there? If not maybe the results wouldn't vary?
- v1ne 2y ago
  Oh, sure there is! https://learn.microsoft.com/en-us/windows-hardware/drivers/network/tcp-ip-offload https://learn.microsoft.com/en-us/windows-hardware/drivers/n...
raggi 2y ago
There are a number of concrete problems: - syscall interfaces are a mess, the primitive APIs are too slow for regular sized packets (~1500 bytes), the overhead is too high. GSO helps but it’s a horrible API, and it’s been buggy even lately due to complexity and poor code standards. - the syscall costs got even higher with spectre mitigation - and this story likely isn’t over. We need a replacement for the BSD sockets / POSIX APIs they’re terrible this decade. Yes, uring is fancy, but there’s a tutorial level API middle ground possible that should be safe and 10x less overhead without resorting to uring level complexity. - system udp buffers are far too small by default - they’re much much smaller than their tcp siblings, essentially no one but experts have been using them, and experts just retune stuff. - udp stack optimizations are possible (such as possible route lookup reuse without connect(2)), gso demonstrates this, though as noted above gso is highly fallible, quite expensive itself, and the design is wholly unnecessarily intricate for what we need, particularly as we want to do this safely from unprivileged userspace. - several optimizations currently available only work at low/mid-scale, such as connect binding to (potentially) avoid route lookups / GSO only being applicable on a socket without high peer-competition (competing peers result in short offload chains due to single-peer constraints, eroding the overhead wins). Despite all this, you can implement GSO and get substantial performance improvements, we (tailscale) have on Linux. There will be a need at some point for platforms to increase platform side buffer sizes for lower end systems, high load/concurrency, bdp and so on, but buffers and congestion control are a high complex and sometimes quite sensitive topic - nonetheless, when you have many applications doing this (presumed future state), there will be a need.
- SomaticPirate 2y ago
  What is GSO?
  - thorncorona 2y ago
    presumably generic segmentation offloading
  - throwaway8481 2y ago
    Generic Segmentation Offload https://www.kernel.org/doc/html/latest/networking/segmentation-offloads.html https://www.kernel.org/doc/html/latest/networking/segmentati...
latentpot 2y ago
QUIC is the standard problem across n number of clients who choose Zscaler and similar content inspection tools. You can block it at the policy level but you also need to have it disabled at the browser level. Which sometimes magically turns on again and leads to a flurry of tickets for 'slow internet', 'Google search not working' etcetera.
- watermelon0 2y ago
  Wouldn't the issue in this case be with Zscaler, and not with QUIC?
- chgs 2y ago
  The problem here is choosing software like zscaler
  - mcosta 2y ago
    Zscaler is not chosen, it is imposed by the corporation
- v1ne 2y ago
  Hmm, interesting. We also have a policies imposed by the Regulator™ that leads to us inspecting all web traffic. All web traffic goes through a proxy that's configured in the web browser. No proxy, no internet. Out of curiosity: What's your use case to use ZScaler for this inspection instead?
  - chgs 2y ago
    You hairpin traffic through your own vpn? How does MITM work with QUIC?
AlphaCharlie 2y ago
Free PDF file of the research: https://arxiv.org/pdf/2310.09423 https://arxiv.org/pdf/2310.09423
jiggawatts 2y ago
I wonder if the trick might be to repurpose technology from server hardware: partition the physical NIC into virtual PCI-e devices with distinct addresses, and map to user-space processes instead of virtual machines. So in essence, each browser tab or even each listening UDP socket could have a distinct IPv6 address dedicated to it, with packets delivered into a ring buffer in user-mode. This is so similar to what goes on with hypervisors now that existing hardware designs might even be able to handle it already. Just an idle thought...
- KaiserPro 2y ago
  Or just have multiple TCP streams. Super simple, low cost, uses all the optimisations we have already. when the latency/packet drop is low, prune the connections and you get monster speed. When the latency/loss is high, grow the number of concurrent connections to overcome slow start. Doesn't give you QUIC like multipath though.
  - m_eiman 2y ago
    There’s Multipath TCP.
    - KaiserPro 2y ago
      I mean there is, but from what I recall its more a link aggregation thing, rather than a network portable system
      - gmokki 2y ago
        It seems to support failover smoothly over various links. Interesting article about combining starlinks, 4G + 5G to single mptcp that works on the move: https://dillonbaird.io/articles/mptcpbonding/ https://dillonbaird.io/articles/mptcpbonding/
- jeroenhd 2y ago
  I've often pondered if it was possible to assign every application/tab/domain/origin a different IPv6 address to exchange data with, to make tracking people just a tad harder, but also to simplify per-process firewall rules. With the bare minimum, a /64, you could easily host billions of addresses per device without running out. I think there may be a limit to how many IP addresses NICs (and maybe drivers) can track at once, though. What I don't really get is why QUIC had to be invented when multi-stream protocols like SCTP already exist. SCTP brings the reliability of TCP with the multi-stream system that makes QUIC good for websites. Piping TLS over it is a bit of a pain (you don't want a separate handshake per stream), but surely there could be techniques to make it less painful (leveraging 0-RTT? Using session resumptions with tickets from the first connected stream?).
wseqyrku 2y ago
There's a work in progress for kernel support: https://github.com/lxin/quic https://github.com/lxin/quic
crashingintoyou 2y ago
Don't have access to the published version but draft at https://arxiv.org/pdf/2310.09423 https://arxiv.org/pdf/2310.09423 mentions ping RTT at 0.23ms. As someone frequently at 150ms+ latency for a lot of websites (and semi-frequently 300ms+ for non-geo-distributed websites), in practice with the latency QUIC is easily the best for throughput, HTTP/1.1 with a decent number of parallel connections is a not-that-distant second, and in a remote third is HTTP/2 due to head-of-line-blocking issues if/when a packet goes missing.
sylware 2y ago
To go faster, you need to simplify a lot.
- bell-cot 2y ago
  To force a lucrative cycle of hardware upgrades, you need software to do the opposite. True story: Back in the early aughties, Intel was hosting regular seminars for dealers and integrators selling either Intel-made PC's, or white box ones. I attended one of those, and the Intel rep openly claimed that Intel had challenged Microsoft to produce software which could bring a GHz CPU to its knees.
larsonnn 2y ago
Site is blocking Apples private relay :(
Banou 2y ago
I think one of the reasons Google choose UDP is that it's already a popular protocol, on which you can build reliable packets, while also having the base UDP unreliability on the side. From my perspective, which is a web developer's, having QUIC, allowed the web standards to easily piggy back on top of it for the Webtransport API, which is ways better than the current HTTP stack and WebRTC which is a complete mess. Basically giving a TCP and UDP implementation for the web. Knowing this, I feel like it makes more sense to me why Google choose this way of doing, which some people seem to be criticizing.
- simoncion 2y ago
  > I think one of the reasons Google choose UDP is that it's already a popular protocol... If you want your packets to reliably travel fairly unmolested between you and an effectively-randomly-chosen-peer on The Greater Internet, you have two transport protocol choices: TCP/IP or UDP/IP. If you don't want the connection-management & etc that TCP/IP does for you, then you have exactly one choice. > ...which some people seem to be criticizing. People are criticizing the fact that on LAN link speeds (and fast (for the US) home internet speeds) QUIC is no better than (and sometimes worse than) previous HTTP transport protocols, despite the large amount of effort put into it. It also seems that some folks are suggesting that Google could have put that time and effort into improving Linux's packet-handling code and (presumably) getting that into both Android and mainline Linux.
dathinab 2y ago
it says it isn't fast _enough_ but as far as I can tell it's fast _enough_ just not as fast as it could be mainly they seem to test situations related to bandwidth/latency which aren't very realistically for the majority of users (because most users don't have supper fast high bandwidth internet) this doesn't meant QUIC can't be faster or we shouldn't look into reducing overhead, just it's likely not as much as a deal as it might initially loook
deleted 2y ago
[deleted]
M2Ys4U 2y ago
>The results show that QUIC and HTTP/2 exhibit similar performance when the network bandwidth is relatively low (below ∼600 Mbps) >Next, we investigate more realistic scenarios by conducting the same file download experiments on major browsers: Chrome, Edge, Firefox, and Opera. We observe that the performance gap is even larger than that in the cURL and quic_client experiments: on Chrome, QUIC begins to fall behind when the bandwidth exceeds ∼500 Mbps. Okay, well, this isn't going to be a problem over the general Internet, it's more of a problem in local networks. For people that have high-speed connections, how often are you getting >500Mbps from a single source?
- sinuhe69 2y ago
  Well, I have other issues with QUIC: when I access Facebook with QUIC, the site often loads the first pages but then it kind of hung, force me to refresh the site, which is annoying. I didn’t know it’s a problem with QUIC, until I turned it off. Since then, FB & Co. load at the same speed, but don’t show this annoying behavior anymore!
- inetknght 2y ago
  > For people that have high-speed connections, how often are you getting >500Mbps from a single source? Often enough over HTTP/1.1 that discussions like this are relevant to my concerns.
- Mali- 2y ago
  Average internet speed has increased by more than 10x in the last 10 years - we've got a few more multipliers in us.
thelastparadise 2y ago
Gotta be QUIC er than that, buddy!
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
throw0101c 2y ago
Netflix has gotten TCP/TLS up to 800 Gbps (over many streams): * https://news.ycombinator.com/item?id=32519881 https://news.ycombinator.com/item?id=32519881 * https://news.ycombinator.com/item?id=33449297 https://news.ycombinator.com/item?id=33449297 hitting 100 Gbps (20k-30k customers) using less that 100W: * https://twitter.com/ocochardlabbe/status/1781848334145130661 https://twitter.com/ocochardlabbe/status/1781848334145130661 * https://news.ycombinator.com/item?id=40630699#unv_40630785 https://news.ycombinator.com/item?id=40630699#unv_40630785
ahmetozer 2y ago
For mobile connectivity -> quic For home internet wifi & cable access -> http2 For heavy loaded enterprise slow wifi network -> quic
necessary 2y ago
Does QUIC do better with packet loss compared to TCP? TCP perceives packet loss as network congestion and so throughput over high bandwidth+high packet loss links suffers.
AtNightWeCode 2y ago
For us, what QUIC solves is that mobile users that move around in the subway and so on are not getting these huge latency spikes. Which was one of our biggest complains.
404mm 2y ago
When looking at the tested browsers, I want to ask why this was not tested on Safari (which is currently the second most used browser by share).
exabrial 2y ago
QUIC needs an unencrypted mode!
- suprjami 2y ago
  Pretty sure Dave Taht would explode if anyone did this.
  - exabrial 2y ago
    spontaneous human combustion… New feature
edwintorok 2y ago
TCP has a lot of offloads that may not all be available for UDP.
wtarreau 2y ago
Something that nobody seems to be talking about here is the congestion control algorithm, which is the problem here. Cubic doesn't like losses. At all. In the kernel, pacing is implemented to minimise losses, allowing Cubic to work acceptably for TCP, but if the network is slightly lossy, the perfs are terrible anyway. QUIC strongly recommends to implement pacing but it's less easy to implement accurately in userland when you have to cross a whole chain than at the queue level in the kernel. Most QUIC implementations use different variations around the protocol to make it behave significantly better, such as preserving the last metrics when facing a loss so that in case it was only a reorder, they can be restored, etc. The article should have compared different server-side implementations, with different settings. We're used to see a ratio of 1:20 in some transatlantic tests. And testing a BBR-enabled QUIC implementation shows tremendous gains compared to TCP with Cubic. Ratios of 1:10 are not uncommon with moderate latency (100ms) and losses (1-3%). At least what QUIC is enlightening is that if TCP has worked so poorly for a very long time (remember that the reason for QUIC was that it was impossible to fix TCP everywhere), it's in large part due to congestion control algorithms, and that since they were implemented in kernel by people carefully reading an academic paper that never considers reality but only in-lab measurements, such algorithms behave pretty poorly in front of the real internet where jitter, reordering, losses, duplicates etc are normal. QUIC allowed many developers to put their fingers in the algos, adjust some thresholds and mechanisms and we're seeing stuff improve fast (it could have improved faster if OpenSSL didn't decide to play against QUIC a few years ago by cowardly refusing to implement the API everyone needed, and imposing to rely on locally-built SSL libs to use QUIC). I'm pretty sure that within 2-3 years, we'll see some of the QUIC improvements ported to TCP, just because QUIC is a great playground to experiment with these algos that for 4 decades had been the reserved territory of just a few people who denied the net as it is and worked for the net how they dreamed it. Look at this for example, it summarizes it all: https://huitema.wordpress.com/2019/11/11/implementing-cubic-congestion-control-in-quic/ https://huitema.wordpress.com/2019/11/11/implementing-cubic-...

664 points by Shank 2y ago

JoshTriplett 2y ago

Seems to be available on arXiv: https://arxiv.org/pdf/2310.09423 https://arxiv.org/pdf/2310.09423

Tempest1981 2y ago
The page headings say "Conference'17, July 2017" -- why is that? Although the sidebar on page 1 shows "13 Oct 2023".
- mrngm 2y ago
  It's likely the authors used an existing conference template to fit in their paper's contents. Upon sending it to the conference, the editors can easily fit the contents in their prescribed format, and the authors know how many characters they can fit in the page limit. arXiv typically contains pre-prints of papers. These may not have been peer-reviewed, and the contents may not reflect the actual "published" paper that was accepted (and/or corrected after peer review) to a conference or journal. arXiv applies a watermark to the submitted PDF such that different versions are distinguishable on download.

jacob019 2y ago

Maybe moving the connection protocol into userspace isn't such a great plan.

foota 2y ago
I don't have access to the article, but they're saying the issue is due to client side ack processing. I suspect they're testing at bandwidths far beyond what's normal for consumer applications.
- dartharva 2y ago
  It's available on arxiv and nope, they are testing mostly for regular 4G/5G speeds. https://arxiv.org/pdf/2310.09423 https://arxiv.org/pdf/2310.09423
  - DannyBee 2y ago
    4g tops out at 1gbps only when one person is on the network. 5g tops out at ~10gbps (some 20gbps i guess) only when one person is on the network. They are testing at 1gbps. This is not regular 4g speed for sure, and it's a rare 5g speed. regular 5g speed is (in the US) 40-50mbps, so, 20x slower than they are testing.
    - dartharva 2y ago
      Still won't be beyond normal consumer applications' capacity, right?
      - DannyBee 2y ago
        correct
    - izend 2y ago
      What about 1gbps fiber at home, it is becoming common in Canada. I have 1gbps up/down.
      - DannyBee 2y ago
        This would affect that. As said, i was only replying to the claim that this affects things at 4g/5g cell phone speeds, which it clearly does not, by their own data.
    - vrighter 2y ago
      Gigabit fiber internet is quite cheap and increasingly available (I'm not from the US). I don't just use the internet over a 4/5g connection. This definitely affects more people than you think.
01HNNWZ0MV43FF 2y ago
Does QUIC mandate that, or is that just the stepping stone until the chicken-and-egg problem is solved and we get kernel support?
- wmf 2y ago
  On mobile the plan is to never use kernel support so that apps can have the latest QUIC on old kernels.
- vlovich123 2y ago
  As others in the thread summarized the paper as saying the issue is ack offload. That has nothing to do with whether the stack is in kernel space or user space. Indeed there’s some concern about this inevitable scenario because the kernel is so slow moving, updates take much longer to propagate to applications needing them without a middle ground whereas as user space stacks they can update as the endpoint applications need them to.
- kmeisthax 2y ago
  No, but it depends on how QUIC works, how Ethernet hardware works, and how much you actually want to offload to the NIC. For example, QUIC has TLS encryption built-in, so anything that's encrypted can't be offloaded. And I don't think most people want to hand all their TLS keys to their NIC[0]. At the very least you probably would have to assign QUIC its own transport, rather than using UDP as "we have raw sockets at home". Problem is, only TCP and UDP reliably traverse the Internet[1]. Everything in the middle is sniffing traffic, messing with options, etc. In fact, Google rejected an alternate transport protocol called SCTP (which does all the stream multiplexing over a single connection that QUIC does) specifically because, among other things, SCTP's a transport protocol and middleboxes choke on it. [0] I am aware that "SSL accelerators" used to do exactly this, but in modern times we have perfectly good crypto accelerators right in our CPU cores. [1] ICMP sometimes traverses the internet, it's how ping works, but a lot of firewalls blackhole ICMP. Or at least they did before IPv6 made it practically mandatory to forward ICMP packets.
  - _flux 2y ago
throwaway984393 2y ago
[dead]
kccqzy 2y ago
The flexibility and ease of changing a userspace protocol IMO far outweighs anything else. If the performance problem described in this article (which I don't have access to) is in userspace QUIC code, it can be fixed and deployed very quickly. If similar performance issue were to be found in TCP, expect to wait multiple years.
- vrighter 2y ago
  Well, the problem is probably that it is in userspace in the first place.
mrweasel 2y ago
Maybe moving the entire application to the browser/cloud wasn't the best idea for a large number of use cases? Video streaming, sure, but we're already able to stream 4K video over a 25Mbit line. With modern internet connections being 200Mbit to 1Gbit, I don't see that we need the bandwidth in private homes. Maybe for video conferencing in large companies, but that also doesn't need to be 4K. The underlying internet protocols are old, so there's no harm in assessing if they've outlived their usefulness. However, we should also consider in web applications and "always connected" is truly the best solution for our day to day application needs.
- kuschku 2y ago
  > With modern internet connections being 200Mbit to 1Gbit, I don't see that we need the bandwidth in private homes Private connections tend to be asymmetrical. In some cases, e.g. old DOCSIS versions, that used to be due to technical necessity. Private connections tend to be unstable, the bandwidth fluctuates quite a bit. Depending on country, the actually guaranteed bandwidth is somewhere between half of what's on the sticker, to nothing at all. Private connections are usually used by families, with multiple people using it at the same time. In recent years, you might have 3+ family members in a video call at the same time. So if you're paying for a 1000/50 line (as is common with DOCSIS deployments), what you're actually getting is usually a 400/20 line that sometimes achieves more. And those 20Mbps upload are now split between multiple people. At the same time, you're absolutely right – Gigabit is enough for most people. Download speeds are enough for quite a while. We should instead be increasing upload speeds and deploying FTTH and IPv6 everywhere to reduce the latency.
  - throwaway2037 2y ago
    This is a great post. I often forget that home Internet connections are frequently shared between many people. This bit: > IPv6 everywhere to reduce the latency I am not an expert on IPv4 vs IPv6. Teach me: How will migrating to IPv6 reduce latency? As I understand, a lot of home Internet connections are always effectively IPv6 via CarrierNAT. (Am I wrong? Or not relevant to your point?)
simiones 2y ago
The problem is that the biggest win by far with QUIC is merging encryption and session negotiation into a single packet, and the kernel teams have been adamant about not wanting to maintain encryption libraries in kernel. So, QUIC or any other protocol like it being in kernel is basically a non-starter.
suprjami 2y ago
Looking forward to QUIC in the kernel. Linux already has kTLS.

JoshTriplett 2y ago

In the early days of QUIC, many people pointed out that the UDP stack has had far far less optimization put into it than the TCP stack. Sure enough, some of the issues identified here arise because the UDP stack isn't doing things that it could do but that nobody has been motivated to make it do, such as UDP generic receive offload. Papers like this are very likely to lead to optimizations both obvious and subtle.

nextaccountic 2y ago
Do you mean that under the same workload, TCP will perform better?
Animats 2y ago
What is UDP offload going to do? UDP barely does anything but queue and copy. Linux scheduling from packet-received to thread has control is not real-time, and if the CPUs are busy, may be rather slow. That's probably part of the bottleneck. The embarrassing thing is that QUIC, even in Google's own benchmarks, only improved performance by about 10%. The added complexity probably isn't worth the trouble. However, it gave Google control of more of the stack, which may have been the real motivation.
- Vecr 2y ago
  I think one of the original drivers was the ability to quickly tweak parameters, after Linux rejected what I think was userspace adjustment of window sizing to be more aggressive than the default. The Linux maintainers didn't want to be responsible for congestion collapse, but UDP lets you spray packets from userspace, so Google went with that.
- JoshTriplett 2y ago
  Among other things, GRO (receive offloading) means you can get more data off of the network card in fewer operations. Linux has receive packet steering, which can help with getting packets from the network card to the right CPU and the right userspace thread without moving from one CPU's cache to another.

mholt 2y ago

I don't have access to the paper but based on the abstract and a quick scan of the presentation, I can confirm that I have seen results like this in Caddy, which enables HTTP/3 out of the box. HTTP/3 implementations vary widely at the moment, and will likely take another decade to optimize to homogeneity. But even then, QUIC requires a lot of state management that TCP doesn't have to worry about (even in the kernel). There's a ton of processing involved with every UDP packet, and small MTUs, still engrained into many middle boxes and even end-user machines these days, don't make it any better. So, yeah, as I felt about QUIC ... oh, about 6 years ago or so... HTTP/2 is actually really quite good enough for most use cases. The far reaches of the world and those without fast connections will benefit, but the majority of global transmissions will likely be best served with HTTP/2. Intuitively, I consider each HTTP major version an increased order of magnitude in complexity. From 1 to 2 the main complexities are binary (that's debatable, since it's technically simpler from an encoding standpoint), compression, and streams; then with HTTP/3 there's _so, so much_ it does to make it work. It _can_ be faster -- that's proven -- but only when networks are slow. TCP congestion control is its own worst enemy, but when networks aren't congested (and with the right algorithm)... guess what. It's fast! And the in-order packet transmissions (head-of-line blocking) makes endpoint code so much simpler and faster. It's no wonder TCP is faster these days when networks are fast. I think servers should offer HTTP/3 but clients should be choosy when to use it, for the sake of their own experience/performance.

deleted 2y ago
[deleted]
altairprime 2y ago
The performance gap is shown to be due to hardware offloading, not due to congestion control, in the arxiv paper above.
- vlovich123 2y ago
  And because Quic is encrypted at a fundamental level, offload likely means needing to share keys with the network card which is a trust concern.

deleted 2y ago

[deleted]

sbstp 2y ago

Even HTTP/2 seems to have been rushed[1]. Chrome has removed support for server push. Maybe more thought should be put into these protocols instead of just rebranding whatever Google is trying to impose on us. [1] https://varnish-cache.org/docs/trunk/phk/h2againagainagain.html https://varnish-cache.org/docs/trunk/phk/h2againagainagain.h...

surajrmal 2y ago
It's okay to make mistakes, that's how you learn and improve. Being conservative has drawbacks of its own. Id argue we need more parties involved earlier in the process rather than just time.
- zdragnar 2y ago
  It's a weird balancing act. On the other hand, waiting for everyone to agree on everything means that the spec will take a decade or two for everyone to come together, and then all the additional time for everyone to actively support it. AJAX is a decent example. Microsoft's Outlook Web Access team implemented XMLHTTP as an activex thing for IE 5 and soon the rest of the vendors adopted it as a standard thing as XmlHttpRequest objects. In fact, I suspect the list of things that exist in browsers because one vendor thought it was a good idea and everyone hopped on board is far, far longer than those designed by committee. Often times, the initially released version is not exactly the same that everyone standardized on, but they all get to build on the real-world consequences of it. I happen to like the TC39 process https://tc39.es/process-document/ https://tc39.es/process-document/ which requires two live implementations with use in the wild for something to get into the final stage and become an official part of the specification. It is obviously harder for something like a network stack than a JavaScript engine to get real world use and feedback, but it has helped to keep a lot of the crazier vendor specific features at bay.
- arcbyte 2y ago
  It's okay to make mistakes, but its not okay to ignore the broad consensus that HTTP2 was TERRIBLY designed and then admit it 10 years later as if it was unknowable. We knew it was bad.

botanical 2y ago

> we identify the root cause to be high receiver-side processing overhead I find this to be the issue when it comes to Google, and I bet it was known before hand; pushing processing to the user. For example, the AV1 video codec was deployed when no consumer had HW decoding capabilities. It saved them on space at the expense of increased CPU usage for the end-user. I don't know what the motive was there; it would still show that they are carbon-neutral while billions are busy processing the data.

anfilt 2y ago
Well I will say if your running servers hit billions of times per day. Offloading processing to the client when safe to do so starts make sense financially. Google does not have to pay for your CPU or storage usage ect... Also I will say if said overhead is not too much it's not that bad of a thing.
kccqzy 2y ago
This is indeed an issue but it's widespread and everyone does it, including Google. Things like servers no longer generating actual dynamic HTML, replaced with servers simply serving pure data like JSON and expecting the client to render it into the DOM. It's not just Google that doesn't care, but the majority of web developers also don't care.
- SquareWheel 2y ago
  There's clearly advantages to writing a web app as an SPA, otherwise web devs wouldn't do it. The idea that web devs "don't care" (about what exactly?) really doesn't make any sense. Moving interactions to JSON in many cases is just a better experience. If you click a Like button on Facebook, which is the better outcome: To see a little animation where the button updates, or for the page to reload with a flash of white, throw away the comment you were part-way through writing, and then scroll you back to the top of the page? There's a reason XMLHttpRequest took the world by storm. More than that, jQuery is still used on more than 80% of websites due in large part to its legacy of making this process easier and cross-browser.

Sparkyte 2y ago

Maybe I'm the only person who thinks that trying to make existing internet protocols faster is wasted energy. But who am I to say anything.

cheema33 2y ago
> Maybe I'm the only person who thinks that trying to make existing internet protocols faster is wasted energy. But who am I to say anything. If you have a valid argument to support your claim, why not present it?
- Sparkyte 2y ago
  They are already expected standards so when you create optimizations you're building on functions that need to be supported additionally on top of them. This leads to incompatibility and sometimes often worse performance as what is being experienced here with QUIC. You can read more about such things from, The Evolution of the Internet Congestion Control. https://groups.csail.mit.edu/ana/Publications/The_Evolution_of_Internet_Congestion_Bauer_Clark_Lehr_TPRC_2009.pdf https://groups.csail.mit.edu/ana/Publications/The_Evolution_... A good solution is to create a newer protocol when the limits of an existing protcol are exceeded. No one thought of needing HTTPS long ago and now we have 443 for HTTP security. If we need something to be faster and it has already achieved an arbitrary limit for the sake of backward compatibility it would be better to introduce a new protocol. I dislike the idea that we're turning into another Reddit where we are pointing fingers at people for updoots. If you dislike my opinion please present one equal to where that can be challenged.
  - likis 2y ago
    You posted your opinion without any kind of accompanying argument, and it was also quite unclear what you meant. Whining about being a target and being downvoted is not really going to help your case. I initially understood your first post as: "Let's not try to make the internet faster" With this reply, you are clarifying your initial post that was very unclear. Now I understand it as: "Let's not try to make existing protocols faster, let's make new protocols instead"

apitman 2y ago

Currently chewing my way laboriously through RFC9000. Definitely concerned by how complex it is. The high level ideas of QUIC seem fairly straight forward, but the spec feels full of edge cases you must account for. Maybe there's no other way, but it makes me uncomfortable. I don't mind too much as long as they never try to take HTTP/1.1 from me.

ironmagma 2y ago
Considering they can’t really even make IPv6 happen, that seems like a likely scenario.
- BartjeD 2y ago
  https://www.google.com/intl/en/ipv6/statistics.html https://www.google.com/intl/en/ipv6/statistics.html I think it's just your little corner of the woods that isn't adopting it. Over here the trend is very clearly to move away from IPv4, except for legacy reasons.
  - apitman 2y ago
    The important milestone is when it's safe to turn IPv4 off. And that's not going to happen as long as any country hasn't fully adopted it, and I don't think that's ever going to happen. For better or worse NAT handles outgoing connections and SNI routing handles incoming connections for most use cases. Self-hosting is the most broken but IMO that's better handled with tunneling anyway so you don't expose your home IP.
    - jeroenhd 2y ago
      IPv4 doesn't need to be off. Hacks and workarounds like DS-Lite can stay with us forever, just like hacks and workarounds like NAT and ALGs will.
      - consp 2y ago
        DS-lite (aka CGNAT), now we don't need to give the costumers a proper IP address anymore. It should be banned as it limits IPv6 adoption and it getting more and more use for "customers own good" and is annoying as hell to work around.

jauntywundrkind 2y ago

I wonder if these results reproduce on Windows. Is there any TCP offload or GSO there? If not maybe the results wouldn't vary?

v1ne 2y ago
Oh, sure there is! https://learn.microsoft.com/en-us/windows-hardware/drivers/network/tcp-ip-offload https://learn.microsoft.com/en-us/windows-hardware/drivers/n...

raggi 2y ago

There are a number of concrete problems: - syscall interfaces are a mess, the primitive APIs are too slow for regular sized packets (~1500 bytes), the overhead is too high. GSO helps but it’s a horrible API, and it’s been buggy even lately due to complexity and poor code standards. - the syscall costs got even higher with spectre mitigation - and this story likely isn’t over. We need a replacement for the BSD sockets / POSIX APIs they’re terrible this decade. Yes, uring is fancy, but there’s a tutorial level API middle ground possible that should be safe and 10x less overhead without resorting to uring level complexity. - system udp buffers are far too small by default - they’re much much smaller than their tcp siblings, essentially no one but experts have been using them, and experts just retune stuff. - udp stack optimizations are possible (such as possible route lookup reuse without connect(2)), gso demonstrates this, though as noted above gso is highly fallible, quite expensive itself, and the design is wholly unnecessarily intricate for what we need, particularly as we want to do this safely from unprivileged userspace. - several optimizations currently available only work at low/mid-scale, such as connect binding to (potentially) avoid route lookups / GSO only being applicable on a socket without high peer-competition (competing peers result in short offload chains due to single-peer constraints, eroding the overhead wins). Despite all this, you can implement GSO and get substantial performance improvements, we (tailscale) have on Linux. There will be a need at some point for platforms to increase platform side buffer sizes for lower end systems, high load/concurrency, bdp and so on, but buffers and congestion control are a high complex and sometimes quite sensitive topic - nonetheless, when you have many applications doing this (presumed future state), there will be a need.

SomaticPirate 2y ago
What is GSO?
- thorncorona 2y ago
  presumably generic segmentation offloading
- throwaway8481 2y ago
  Generic Segmentation Offload https://www.kernel.org/doc/html/latest/networking/segmentation-offloads.html https://www.kernel.org/doc/html/latest/networking/segmentati...

latentpot 2y ago

QUIC is the standard problem across n number of clients who choose Zscaler and similar content inspection tools. You can block it at the policy level but you also need to have it disabled at the browser level. Which sometimes magically turns on again and leads to a flurry of tickets for 'slow internet', 'Google search not working' etcetera.

watermelon0 2y ago
Wouldn't the issue in this case be with Zscaler, and not with QUIC?
chgs 2y ago
The problem here is choosing software like zscaler
- mcosta 2y ago
  Zscaler is not chosen, it is imposed by the corporation
v1ne 2y ago
Hmm, interesting. We also have a policies imposed by the Regulator™ that leads to us inspecting all web traffic. All web traffic goes through a proxy that's configured in the web browser. No proxy, no internet. Out of curiosity: What's your use case to use ZScaler for this inspection instead?
- chgs 2y ago
  You hairpin traffic through your own vpn? How does MITM work with QUIC?

AlphaCharlie 2y ago

Free PDF file of the research: https://arxiv.org/pdf/2310.09423 https://arxiv.org/pdf/2310.09423

jiggawatts 2y ago

I wonder if the trick might be to repurpose technology from server hardware: partition the physical NIC into virtual PCI-e devices with distinct addresses, and map to user-space processes instead of virtual machines. So in essence, each browser tab or even each listening UDP socket could have a distinct IPv6 address dedicated to it, with packets delivered into a ring buffer in user-mode. This is so similar to what goes on with hypervisors now that existing hardware designs might even be able to handle it already. Just an idle thought...

KaiserPro 2y ago
Or just have multiple TCP streams. Super simple, low cost, uses all the optimisations we have already. when the latency/packet drop is low, prune the connections and you get monster speed. When the latency/loss is high, grow the number of concurrent connections to overcome slow start. Doesn't give you QUIC like multipath though.
- m_eiman 2y ago
  There’s Multipath TCP.
  - KaiserPro 2y ago
    I mean there is, but from what I recall its more a link aggregation thing, rather than a network portable system
    - gmokki 2y ago
      It seems to support failover smoothly over various links. Interesting article about combining starlinks, 4G + 5G to single mptcp that works on the move: https://dillonbaird.io/articles/mptcpbonding/ https://dillonbaird.io/articles/mptcpbonding/
jeroenhd 2y ago
I've often pondered if it was possible to assign every application/tab/domain/origin a different IPv6 address to exchange data with, to make tracking people just a tad harder, but also to simplify per-process firewall rules. With the bare minimum, a /64, you could easily host billions of addresses per device without running out. I think there may be a limit to how many IP addresses NICs (and maybe drivers) can track at once, though. What I don't really get is why QUIC had to be invented when multi-stream protocols like SCTP already exist. SCTP brings the reliability of TCP with the multi-stream system that makes QUIC good for websites. Piping TLS over it is a bit of a pain (you don't want a separate handshake per stream), but surely there could be techniques to make it less painful (leveraging 0-RTT? Using session resumptions with tickets from the first connected stream?).

wseqyrku 2y ago

There's a work in progress for kernel support: https://github.com/lxin/quic https://github.com/lxin/quic

crashingintoyou 2y ago

Don't have access to the published version but draft at https://arxiv.org/pdf/2310.09423 https://arxiv.org/pdf/2310.09423 mentions ping RTT at 0.23ms. As someone frequently at 150ms+ latency for a lot of websites (and semi-frequently 300ms+ for non-geo-distributed websites), in practice with the latency QUIC is easily the best for throughput, HTTP/1.1 with a decent number of parallel connections is a not-that-distant second, and in a remote third is HTTP/2 due to head-of-line-blocking issues if/when a packet goes missing.

sylware 2y ago

To go faster, you need to simplify a lot.

bell-cot 2y ago
To force a lucrative cycle of hardware upgrades, you need software to do the opposite. True story: Back in the early aughties, Intel was hosting regular seminars for dealers and integrators selling either Intel-made PC's, or white box ones. I attended one of those, and the Intel rep openly claimed that Intel had challenged Microsoft to produce software which could bring a GHz CPU to its knees.

larsonnn 2y ago

Site is blocking Apples private relay :(

Banou 2y ago

I think one of the reasons Google choose UDP is that it's already a popular protocol, on which you can build reliable packets, while also having the base UDP unreliability on the side. From my perspective, which is a web developer's, having QUIC, allowed the web standards to easily piggy back on top of it for the Webtransport API, which is ways better than the current HTTP stack and WebRTC which is a complete mess. Basically giving a TCP and UDP implementation for the web. Knowing this, I feel like it makes more sense to me why Google choose this way of doing, which some people seem to be criticizing.

simoncion 2y ago
> I think one of the reasons Google choose UDP is that it's already a popular protocol... If you want your packets to reliably travel fairly unmolested between you and an effectively-randomly-chosen-peer on The Greater Internet, you have two transport protocol choices: TCP/IP or UDP/IP. If you don't want the connection-management & etc that TCP/IP does for you, then you have exactly one choice. > ...which some people seem to be criticizing. People are criticizing the fact that on LAN link speeds (and fast (for the US) home internet speeds) QUIC is no better than (and sometimes worse than) previous HTTP transport protocols, despite the large amount of effort put into it. It also seems that some folks are suggesting that Google could have put that time and effort into improving Linux's packet-handling code and (presumably) getting that into both Android and mainline Linux.

dathinab 2y ago

it says it isn't fast _enough_ but as far as I can tell it's fast _enough_ just not as fast as it could be mainly they seem to test situations related to bandwidth/latency which aren't very realistically for the majority of users (because most users don't have supper fast high bandwidth internet) this doesn't meant QUIC can't be faster or we shouldn't look into reducing overhead, just it's likely not as much as a deal as it might initially loook

deleted 2y ago

[deleted]

M2Ys4U 2y ago

>The results show that QUIC and HTTP/2 exhibit similar performance when the network bandwidth is relatively low (below ∼600 Mbps) >Next, we investigate more realistic scenarios by conducting the same file download experiments on major browsers: Chrome, Edge, Firefox, and Opera. We observe that the performance gap is even larger than that in the cURL and quic_client experiments: on Chrome, QUIC begins to fall behind when the bandwidth exceeds ∼500 Mbps. Okay, well, this isn't going to be a problem over the general Internet, it's more of a problem in local networks. For people that have high-speed connections, how often are you getting >500Mbps from a single source?

sinuhe69 2y ago
Well, I have other issues with QUIC: when I access Facebook with QUIC, the site often loads the first pages but then it kind of hung, force me to refresh the site, which is annoying. I didn’t know it’s a problem with QUIC, until I turned it off. Since then, FB & Co. load at the same speed, but don’t show this annoying behavior anymore!
inetknght 2y ago
> For people that have high-speed connections, how often are you getting >500Mbps from a single source? Often enough over HTTP/1.1 that discussions like this are relevant to my concerns.
Mali- 2y ago
Average internet speed has increased by more than 10x in the last 10 years - we've got a few more multipliers in us.

thelastparadise 2y ago

Gotta be QUIC er than that, buddy!

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]

throw0101c 2y ago

Netflix has gotten TCP/TLS up to 800 Gbps (over many streams): * https://news.ycombinator.com/item?id=32519881 https://news.ycombinator.com/item?id=32519881 * https://news.ycombinator.com/item?id=33449297 https://news.ycombinator.com/item?id=33449297 hitting 100 Gbps (20k-30k customers) using less that 100W: * https://twitter.com/ocochardlabbe/status/1781848334145130661 https://twitter.com/ocochardlabbe/status/1781848334145130661 * https://news.ycombinator.com/item?id=40630699#unv_40630785 https://news.ycombinator.com/item?id=40630699#unv_40630785

ahmetozer 2y ago

For mobile connectivity -> quic For home internet wifi & cable access -> http2 For heavy loaded enterprise slow wifi network -> quic

necessary 2y ago

Does QUIC do better with packet loss compared to TCP? TCP perceives packet loss as network congestion and so throughput over high bandwidth+high packet loss links suffers.

AtNightWeCode 2y ago

For us, what QUIC solves is that mobile users that move around in the subway and so on are not getting these huge latency spikes. Which was one of our biggest complains.

404mm 2y ago

When looking at the tested browsers, I want to ask why this was not tested on Safari (which is currently the second most used browser by share).

exabrial 2y ago

QUIC needs an unencrypted mode!

suprjami 2y ago
Pretty sure Dave Taht would explode if anyone did this.
- exabrial 2y ago
  spontaneous human combustion… New feature

edwintorok 2y ago

TCP has a lot of offloads that may not all be available for UDP.

wtarreau 2y ago

Something that nobody seems to be talking about here is the congestion control algorithm, which is the problem here. Cubic doesn't like losses. At all. In the kernel, pacing is implemented to minimise losses, allowing Cubic to work acceptably for TCP, but if the network is slightly lossy, the perfs are terrible anyway. QUIC strongly recommends to implement pacing but it's less easy to implement accurately in userland when you have to cross a whole chain than at the queue level in the kernel. Most QUIC implementations use different variations around the protocol to make it behave significantly better, such as preserving the last metrics when facing a loss so that in case it was only a reorder, they can be restored, etc. The article should have compared different server-side implementations, with different settings. We're used to see a ratio of 1:20 in some transatlantic tests. And testing a BBR-enabled QUIC implementation shows tremendous gains compared to TCP with Cubic. Ratios of 1:10 are not uncommon with moderate latency (100ms) and losses (1-3%). At least what QUIC is enlightening is that if TCP has worked so poorly for a very long time (remember that the reason for QUIC was that it was impossible to fix TCP everywhere), it's in large part due to congestion control algorithms, and that since they were implemented in kernel by people carefully reading an academic paper that never considers reality but only in-lab measurements, such algorithms behave pretty poorly in front of the real internet where jitter, reordering, losses, duplicates etc are normal. QUIC allowed many developers to put their fingers in the algos, adjust some thresholds and mechanisms and we're seeing stuff improve fast (it could have improved faster if OpenSSL didn't decide to play against QUIC a few years ago by cowardly refusing to implement the API everyone needed, and imposing to rely on locally-built SSL libs to use QUIC). I'm pretty sure that within 2-3 years, we'll see some of the QUIC improvements ported to TCP, just because QUIC is a great playground to experiment with these algos that for 4 decades had been the reserved territory of just a few people who denied the net as it is and worked for the net how they dreamed it. Look at this for example, it summarizes it all: https://huitema.wordpress.com/2019/11/11/implementing-cubic-congestion-control-in-quic/ https://huitema.wordpress.com/2019/11/11/implementing-cubic-...