4 ms·

What nobody talks about is the lack of server-side offloads for QUIC. Things like TSO, LRO, and even hardware offloaded kTLS. Without those offloads, I estimat

What nobody talks about is the lack of server-side offloads for QUIC. Things like TSO, LRO, and even hardware offloaded kTLS. Without those offloads, I estimate I'd be lucky to get 200Gb/s out of the same Netflix CDN server hardware that can serve TLS-encrypted TCP at over 700Gb/s. Do the benefits of QUIC really justify the economic and environmental impacts of that kind of loss of inefficiency on the server side? And yes, I know that some of these offloads are being worked on, but they are not here today.

miohtama 4y ago
A good question. Here is 2021 study: “Takeaway: We observe that QUIC provides significant improvements over TLS/TCP in low-bandwidth and high-RTT regions for video downloads. QUIC handshakes towards YouTube media servers offer an improvement of 534 ms (IN) and 406 ms (DE) when compared with TLS/TCP. We also observe that the overall download rate for TLS/TCP is higher than QUIC partly due to kernel optimizations such as LRO available for the TCP stack. QUIC provides a better video streaming experience with a lesser number and duration of stall events compared to TLS/TCP. We observe that TLS/TCP exhibits up to 50% longer stall durations compared to QUIC at 50th percentile for high loss networks.“ https://vaibhavbajpai.com/documents/papers/proceedings/quic-tnsm-2022.pdf https://vaibhavbajpai.com/documents/papers/proceedings/quic-... Also the study discussed about the server-side CPU usage. If you are worried about server-side offloading, you can still stick to TCP/IP until the hardware acceleration and other offloading catches up.
- drewg123 4y ago
  I skimmed the paper, but I could not determine what TCP congestion control method they were testing against. Given the stall time, I expect that it was not BBR (which is the congestion control used by QUIC). If you improve TCP's congestion control and can also control the client TCP stack, most of BBR's advantages over TCP disappear.
  - matsur 4y ago
    QUIC doesn't imply BBR as congestion control. eg. Our (Cloudflare's) QUIC implementation supports multiple algorithms. https://blog.cloudflare.com/cubic-and-hystart-support-in-quiche/ https://blog.cloudflare.com/cubic-and-hystart-support-in-qui...
    - miohtama 4y ago
      Very interesting post! Do you have any sources what is the true packet loss % on different real world Internet connections?
      - matsur 4y ago
        We have some data :) I’ll see if we can get a blog together.
iforgotpassword 4y ago
I'd say for most use cases, TCP will be fine for a long time. QUIC has two things going for it: The Google (web) use-case where you load a bazillion resources for your website in parallel, and want to employ some manually determined prioritization of what should load first, ie. the parts that make the site look complete before some script doing random things in the footer. And ads. The other advantage is that thanks to being based on UDP, you get a STUNable version of TCP, which is really great for p2p apps like Syncthing. For making sense in the data center, or for delivering bulk data, QUIC just doesn't have a killer feature really, compared to the ages old battle-tested TCP.
sbierwagen 4y ago
Some of Google's first ASICs were put into their in-house routers, more than a decade ago now. One wonders if they don't already have NICs with QUIC offload.

miohtama 4y ago

A good question. Here is 2021 study: “Takeaway: We observe that QUIC provides significant improvements over TLS/TCP in low-bandwidth and high-RTT regions for video downloads. QUIC handshakes towards YouTube media servers offer an improvement of 534 ms (IN) and 406 ms (DE) when compared with TLS/TCP. We also observe that the overall download rate for TLS/TCP is higher than QUIC partly due to kernel optimizations such as LRO available for the TCP stack. QUIC provides a better video streaming experience with a lesser number and duration of stall events compared to TLS/TCP. We observe that TLS/TCP exhibits up to 50% longer stall durations compared to QUIC at 50th percentile for high loss networks.“ https://vaibhavbajpai.com/documents/papers/proceedings/quic-tnsm-2022.pdf https://vaibhavbajpai.com/documents/papers/proceedings/quic-... Also the study discussed about the server-side CPU usage. If you are worried about server-side offloading, you can still stick to TCP/IP until the hardware acceleration and other offloading catches up.

drewg123 4y ago
I skimmed the paper, but I could not determine what TCP congestion control method they were testing against. Given the stall time, I expect that it was not BBR (which is the congestion control used by QUIC). If you improve TCP's congestion control and can also control the client TCP stack, most of BBR's advantages over TCP disappear.
- matsur 4y ago
  QUIC doesn't imply BBR as congestion control. eg. Our (Cloudflare's) QUIC implementation supports multiple algorithms. https://blog.cloudflare.com/cubic-and-hystart-support-in-quiche/ https://blog.cloudflare.com/cubic-and-hystart-support-in-qui...
  - miohtama 4y ago
    Very interesting post! Do you have any sources what is the true packet loss % on different real world Internet connections?
    - matsur 4y ago
      We have some data :) I’ll see if we can get a blog together.

iforgotpassword 4y ago

I'd say for most use cases, TCP will be fine for a long time. QUIC has two things going for it: The Google (web) use-case where you load a bazillion resources for your website in parallel, and want to employ some manually determined prioritization of what should load first, ie. the parts that make the site look complete before some script doing random things in the footer. And ads. The other advantage is that thanks to being based on UDP, you get a STUNable version of TCP, which is really great for p2p apps like Syncthing. For making sense in the data center, or for delivering bulk data, QUIC just doesn't have a killer feature really, compared to the ages old battle-tested TCP.

sbierwagen 4y ago

Some of Google's first ASICs were put into their in-house routers, more than a decade ago now. One wonders if they don't already have NICs with QUIC offload.