7 ms·

Curl creator/maintainer Daniel Stenberg blogged about HTTP/3 in curl a few months ago: https://daniel.haxx.se/blog/2024/06/10/http-3-in-curl-mid-2024/ https://d

Curl creator/maintainer Daniel Stenberg blogged about HTTP/3 in curl a few months ago: https://daniel.haxx.se/blog/2024/06/10/http-3-in-curl-mid-2024/ https://daniel.haxx.se/blog/2024/06/10/http-3-in-curl-mid-20... One of the things he highlighted was the higher CPU utilization of HTTP/3, to the point where CPU can limit throughput. I wonder how much of this is due to the immaturity of the implementations, and how much this is inherit due to way QUIC was designed?

cj 2y ago
I’ve always been under the impression that QUIC was designed for connections that aren’t guaranteed to be stable or fast. Like mobile networks. I never got the impression that it was intended to make all connections faster. If viewed from that perspective, the tradeoffs make sense. Although I’m no expert and encourage someone with more knowledge to correct me.
- dan-robertson 2y ago
  I think that’s a pretty good impression. Lots of features for those cases: - better behaviour under packet loss (you don’t need to read byte n before you can see byte n+1 like in tcp) - better behaviour under client ip changes (which happen when switching between cellular data and wifi) - moving various tricks for getting good latency and throughput in the real world into user space (things like pacing, bbr) and not leaving enough unencrypted information in packets for middleware boxes to get too funky
- therealmarv 2y ago
  It makes everything faster, it's an evolvement of HTTP/2 in many ways. I recommend watching https://www.youtube.com/watch?v=cdb7M37o9sU https://www.youtube.com/watch?v=cdb7M37o9sU
- fulafel 2y ago
  That's how the internet works, there's no guaranteed delivery and TCP bandwidth estimation is based on when packets start to be dropped when you send too many.
paulddraper 2y ago
Those performance results surprised me too. His testing has CPU-bound quiche at <200MB/s and nghttp2 was >900MB/s. I wonder if the CPU was throttled. Because if HTTP 3 impl took 4x CPU that could be interesting but not necessarily a big problem if the absolute value was very low to begin with.
dan-robertson 2y ago
Two recommendations are for improving receiver-side implementations – optimising them and making them multithreaded. Those suggest some immaturity of the implementations. A third recommendation is UDP GRO, which means modifying kernels and ideally NIC hardware to group received UDP packets together in a way that reduces per-packet work (you do lots of per-group work instead of per-packet work). This already exists in TCP and there are similar things on the send side (eg TSO, GSO in Linux), and feels a bit like immaturity but maybe harder to remedy considering the potential lack of hardware capabilities. The abstract talks about the cost of how acks work in QUIC but I didn’t look into that claim. Another feature you see for modern tcp-based servers is offloading tls to the hardware. I think this matters more for servers that may have many concurrent tcp streams to send. On Linux you can get this either with userspace networking or by doing ‘kernel tls’ which will offload to hardware if possible. That feature also exists for some funny stuff in Linux about breaking down a tcp stream into ‘messages’ which can be sent to different threads, though I don’t know if it allows eagerly passing some later messages when earlier packets were lost.
therealmarv 2y ago
"immaturity of the implementations" is a funny wording here. QUIC was created because there is absolutely NO WAY that all internet hardware (including all middleware etc) out there will support a new TCP or TLS standard. So QUIC is an elegant solution to get a new transport standard on top of legacy internet hardware (on top of UDP). In an ideal World we would create a new TCP and TLS standard and replace and/or update all internet routers and hardware everywhere World Wide so that it is implemented with less CPU utilization ;)
- api 2y ago
  A major mistake in IP’s design was to allow middle boxes. The protocol should have had some kind of minimal header auth feature to intentionally break them. It wouldn’t have to be strong crypto, just enough to make middle boxes impractical. It would have forced IPv6 migration immediately (no NAT) and forced endpoints to be secured with local firewalls and better software instead of middle boxes. The Internet would be so much simpler, faster, and more capable. Peer to peer would be trivial. Everything would just work. Protocol innovation would be possible. Of course tech is full of better roads not taken. We are prisoners of network effects and accidents of history freezing ugly hacks into place.
  - dcow 2y ago
    Now that’s a horse of a different color! I’m already opining this alt reality. Middle-boxes and everyone touching them ruined the internet.
  - johncolanduoni 2y ago
    Making IPv4 headers resistant to tampering wouldn't have helped with IPv6 rollout, as routers (both customer and ISP) would still need to be updated to be able to understand how to route packets with the new headers.
    - ajb 2y ago
      The GP's point is that if middle boxes couldn't rewrite the header, NAt would be impossible. And if NAT were impossible, ipV4 would have died several years ago because NAT allowed more computers than addresses.

cj 2y ago

I’ve always been under the impression that QUIC was designed for connections that aren’t guaranteed to be stable or fast. Like mobile networks. I never got the impression that it was intended to make all connections faster. If viewed from that perspective, the tradeoffs make sense. Although I’m no expert and encourage someone with more knowledge to correct me.

dan-robertson 2y ago
I think that’s a pretty good impression. Lots of features for those cases: - better behaviour under packet loss (you don’t need to read byte n before you can see byte n+1 like in tcp) - better behaviour under client ip changes (which happen when switching between cellular data and wifi) - moving various tricks for getting good latency and throughput in the real world into user space (things like pacing, bbr) and not leaving enough unencrypted information in packets for middleware boxes to get too funky
therealmarv 2y ago
It makes everything faster, it's an evolvement of HTTP/2 in many ways. I recommend watching https://www.youtube.com/watch?v=cdb7M37o9sU https://www.youtube.com/watch?v=cdb7M37o9sU
fulafel 2y ago
That's how the internet works, there's no guaranteed delivery and TCP bandwidth estimation is based on when packets start to be dropped when you send too many.

paulddraper 2y ago

Those performance results surprised me too. His testing has CPU-bound quiche at <200MB/s and nghttp2 was >900MB/s. I wonder if the CPU was throttled. Because if HTTP 3 impl took 4x CPU that could be interesting but not necessarily a big problem if the absolute value was very low to begin with.

dan-robertson 2y ago

Two recommendations are for improving receiver-side implementations – optimising them and making them multithreaded. Those suggest some immaturity of the implementations. A third recommendation is UDP GRO, which means modifying kernels and ideally NIC hardware to group received UDP packets together in a way that reduces per-packet work (you do lots of per-group work instead of per-packet work). This already exists in TCP and there are similar things on the send side (eg TSO, GSO in Linux), and feels a bit like immaturity but maybe harder to remedy considering the potential lack of hardware capabilities. The abstract talks about the cost of how acks work in QUIC but I didn’t look into that claim. Another feature you see for modern tcp-based servers is offloading tls to the hardware. I think this matters more for servers that may have many concurrent tcp streams to send. On Linux you can get this either with userspace networking or by doing ‘kernel tls’ which will offload to hardware if possible. That feature also exists for some funny stuff in Linux about breaking down a tcp stream into ‘messages’ which can be sent to different threads, though I don’t know if it allows eagerly passing some later messages when earlier packets were lost.

therealmarv 2y ago

"immaturity of the implementations" is a funny wording here. QUIC was created because there is absolutely NO WAY that all internet hardware (including all middleware etc) out there will support a new TCP or TLS standard. So QUIC is an elegant solution to get a new transport standard on top of legacy internet hardware (on top of UDP). In an ideal World we would create a new TCP and TLS standard and replace and/or update all internet routers and hardware everywhere World Wide so that it is implemented with less CPU utilization ;)

api 2y ago
A major mistake in IP’s design was to allow middle boxes. The protocol should have had some kind of minimal header auth feature to intentionally break them. It wouldn’t have to be strong crypto, just enough to make middle boxes impractical. It would have forced IPv6 migration immediately (no NAT) and forced endpoints to be secured with local firewalls and better software instead of middle boxes. The Internet would be so much simpler, faster, and more capable. Peer to peer would be trivial. Everything would just work. Protocol innovation would be possible. Of course tech is full of better roads not taken. We are prisoners of network effects and accidents of history freezing ugly hacks into place.
- dcow 2y ago
  Now that’s a horse of a different color! I’m already opining this alt reality. Middle-boxes and everyone touching them ruined the internet.
- johncolanduoni 2y ago
  Making IPv4 headers resistant to tampering wouldn't have helped with IPv6 rollout, as routers (both customer and ISP) would still need to be updated to be able to understand how to route packets with the new headers.
  - ajb 2y ago
    The GP's point is that if middle boxes couldn't rewrite the header, NAt would be impossible. And if NAT were impossible, ipV4 would have died several years ago because NAT allowed more computers than addresses.