7 ms·

What is UDP offload going to do? UDP barely does anything but queue and copy. Linux scheduling from packet-received to thread has control is not real-time, and

What is UDP offload going to do? UDP barely does anything but queue and copy. Linux scheduling from packet-received to thread has control is not real-time, and if the CPUs are busy, may be rather slow. That's probably part of the bottleneck. The embarrassing thing is that QUIC, even in Google's own benchmarks, only improved performance by about 10%. The added complexity probably isn't worth the trouble. However, it gave Google control of more of the stack, which may have been the real motivation.

Vecr 2y ago
I think one of the original drivers was the ability to quickly tweak parameters, after Linux rejected what I think was userspace adjustment of window sizing to be more aggressive than the default. The Linux maintainers didn't want to be responsible for congestion collapse, but UDP lets you spray packets from userspace, so Google went with that.
JoshTriplett 2y ago
Among other things, GRO (receive offloading) means you can get more data off of the network card in fewer operations. Linux has receive packet steering, which can help with getting packets from the network card to the right CPU and the right userspace thread without moving from one CPU's cache to another.
- suprjami 2y ago
  RPS is just software RSS. You mean Receive Flow Steering, and RFS can only control RPS, so to do it in hardware you actually mean Accelerated RFS (which requires a pretty fancy NIC these days). Even ignoring the hardware requirement, unfortunately it's not that simple. I find results vary wildly whether you should put process and softirq on the same CPU core (sharing L1 and L2) or just on the same CPU socket (sharing L3 but don't constantly blow out L1/L2). Eric Dumazet said years ago at a Netdev.conf that L1 cache sizes have really not kept up with reality. That matches my experience. QUIC doing so much in userspace adds another class of application which has a so-far uncommon design pattern. I don't think it's possible to say whether any QUIC application benefits from RFS or not.
infogulch 2y ago
In my head the main benefit of QUIC was always multipath, aka the ability to switch interfaces on demand without losing the connection. There's MPTCP but who knows how viable it is.
- rocqua 2y ago
  Mptcp sees use in the Telco space, so they probably know.
- modeless 2y ago
  Is that actually implemented and working in practice? My connection still hangs whenever my wifi goes out of range...
- Sesse__ 2y ago
  Apple's Siri is using MPTCP, so it is presumably viable.
  - jshier 2y ago
    It requires explicit backend support, and Apple supports it for many of their services, but I've never seen another public API that does. Anyone have any examples?
    - mh- 2y ago
      Last I looked into this (many years), ELB/GLBs didn't support it on AWS/GCP respectively. That prevented us from further considering implementing it at the time (mobile app -> AWS-hosted EC2 instances behind an ELB). Not sure if that's changed, but at the time it wasn't worth having to consider rolling our own LBs. To answer your original question, no, I haven't (knowingly) seen it on any public APIs.
- suprjami 2y ago
  I always thought the main benefit of QUIC was to encrypt the important part of the transport header, so endpoints control their own destiny, not some middle device. If I had a dollar for every firewall vendor who thought dropping TCP retransmissions or TCP Reset was a good idea...
amluto 2y ago
Last I looked (several months ago), Linux's UDP stack did not seemed well tuned in its memory management accounting. For background, the mental model of what receiving network data looks like in userspace is almost completely backwards compared to how general-purpose kernel network receive actually works. User code thinks it allocates a buffer (per-socket or perhaps a fancier io_uring scheme), then receives packets into that buffer, then processes them. The kernel is the other way around. The kernel allocates buffers and feeds pointers to those buffers to the NIC. The NIC receives packets and DMAs them into the buffers, then tells the kernel. But the NIC and the kernel have absolutely no concept of which socket those buffers belong to until after they are DMAed into the buffers. So the kernel cannot possibly map received packets to the actual recipient's memory. So instead, after identifying who owns a received packet, the kernel retroactively charges the recipient for the memory. This happens on a per-packet basis, it involves per-socket and cgroup accounting, and there is no support for having a socket "pre-allocate" this memory in advance of receiving a packet. So the accounting is gnarly, involves atomic operations, and seems quite unlikely to win any speed awards. On a very cursory inspection, the TCP code seemed better tuned, and it possibly also won by generally handling more bytes per operation. Keep in mind that the kernel can't copy data to application memory synchronously -- the application memory might be paged out when a packet shows up. So instead the whole charging dance above happens immediately when a packet is received, and the data is copied later on. For quite a long time, I've thought it would be nifty if there was a NIC that kept received data in its own RAM and then allowed it to be efficiently DMAed to application memory when the application was ready for it. In essence, a lot of the accounting and memory management logic could move out of the kernel into the NIC. I'm not aware of anyone doing this.
- fragmede 2y ago
  RDMA is common for high performance applications but it doesn't work over the Internet.
  - Danieru 2y ago
    It's a good thing the NIC is connected over pcie then.
  - 2y ago
apitman 2y ago
Ditching head of line blocking is potentially a big win, but I really wish it wouldn't have come with so much complexity.
raggi 2y ago
UDP offload gets you implicitly today: - 64 packets per syscall, which is enough data to amortize the syscall overhead - a single packet is not. - UDP offload optionally lets you defer checksum computation, often offloading it to hardware. - UDP offload lets you skip/reuse route lookups for subsequent packets in a bundle. What UDP offload is no good for though, is large scale servers - the current APIs only work when the incoming packet chains neatly organize into batches per peer socket. If you have many thousands of active sockets you’ll stop having full bundles and the overhead starts sneaking back in. As I said in another thread, we really need a replacement for the BSD APIs here, they just don’t scale for modern hardware constraints and software needs - much too expensive per packet.
10000truths 2y ago
Bulk throughout isn't on par with TLS mainly because NICs with dedicated hardware for QUIC offload aren't commercially available (yet). Latency is undoubtedly better - the 1-RTT QUIC handshake substantially reduces time-to-first-byte compared to TLS.
majke 2y ago
> What is UDP offload going to do? Handling ACK packets in kernelspace would be one thing - helping for example RTT estimation. With userspace stack ACK's are handled in application and are subject to scheduler, suffering a lot on a loaded system.
- morning-coffee 2y ago
  There are no ACKs inherent in the UDP protocol, so "UDP offload" is not where the savings are. There are ACKs in the QUIC protocol and they are carried by UDP datagrams which need to make their way up to user land to be processed, and this is the crux of the issue. What is needed is for QUIC offload to be invented/supported by HW so that most of the high-frequency/tiny-packet processing happens there, just as it does today for TCP offload. TCP large-send and large-receive offload is what is responsible for all the CPU savings as the application deals in 64KB or larger send/receives and the segmentation and receive coalescing all happen in hardware before an interrupt is even generated to involve the kernel, let alone userland.

Vecr 2y ago

I think one of the original drivers was the ability to quickly tweak parameters, after Linux rejected what I think was userspace adjustment of window sizing to be more aggressive than the default. The Linux maintainers didn't want to be responsible for congestion collapse, but UDP lets you spray packets from userspace, so Google went with that.

JoshTriplett 2y ago

Among other things, GRO (receive offloading) means you can get more data off of the network card in fewer operations. Linux has receive packet steering, which can help with getting packets from the network card to the right CPU and the right userspace thread without moving from one CPU's cache to another.

suprjami 2y ago
RPS is just software RSS. You mean Receive Flow Steering, and RFS can only control RPS, so to do it in hardware you actually mean Accelerated RFS (which requires a pretty fancy NIC these days). Even ignoring the hardware requirement, unfortunately it's not that simple. I find results vary wildly whether you should put process and softirq on the same CPU core (sharing L1 and L2) or just on the same CPU socket (sharing L3 but don't constantly blow out L1/L2). Eric Dumazet said years ago at a Netdev.conf that L1 cache sizes have really not kept up with reality. That matches my experience. QUIC doing so much in userspace adds another class of application which has a so-far uncommon design pattern. I don't think it's possible to say whether any QUIC application benefits from RFS or not.

infogulch 2y ago

In my head the main benefit of QUIC was always multipath, aka the ability to switch interfaces on demand without losing the connection. There's MPTCP but who knows how viable it is.

rocqua 2y ago
Mptcp sees use in the Telco space, so they probably know.
modeless 2y ago
Is that actually implemented and working in practice? My connection still hangs whenever my wifi goes out of range...
Sesse__ 2y ago
Apple's Siri is using MPTCP, so it is presumably viable.
- jshier 2y ago
  It requires explicit backend support, and Apple supports it for many of their services, but I've never seen another public API that does. Anyone have any examples?
  - mh- 2y ago
    Last I looked into this (many years), ELB/GLBs didn't support it on AWS/GCP respectively. That prevented us from further considering implementing it at the time (mobile app -> AWS-hosted EC2 instances behind an ELB). Not sure if that's changed, but at the time it wasn't worth having to consider rolling our own LBs. To answer your original question, no, I haven't (knowingly) seen it on any public APIs.
suprjami 2y ago
I always thought the main benefit of QUIC was to encrypt the important part of the transport header, so endpoints control their own destiny, not some middle device. If I had a dollar for every firewall vendor who thought dropping TCP retransmissions or TCP Reset was a good idea...

amluto 2y ago

Last I looked (several months ago), Linux's UDP stack did not seemed well tuned in its memory management accounting. For background, the mental model of what receiving network data looks like in userspace is almost completely backwards compared to how general-purpose kernel network receive actually works. User code thinks it allocates a buffer (per-socket or perhaps a fancier io_uring scheme), then receives packets into that buffer, then processes them. The kernel is the other way around. The kernel allocates buffers and feeds pointers to those buffers to the NIC. The NIC receives packets and DMAs them into the buffers, then tells the kernel. But the NIC and the kernel have absolutely no concept of which socket those buffers belong to until after they are DMAed into the buffers. So the kernel cannot possibly map received packets to the actual recipient's memory. So instead, after identifying who owns a received packet, the kernel retroactively charges the recipient for the memory. This happens on a per-packet basis, it involves per-socket and cgroup accounting, and there is no support for having a socket "pre-allocate" this memory in advance of receiving a packet. So the accounting is gnarly, involves atomic operations, and seems quite unlikely to win any speed awards. On a very cursory inspection, the TCP code seemed better tuned, and it possibly also won by generally handling more bytes per operation. Keep in mind that the kernel can't copy data to application memory synchronously -- the application memory might be paged out when a packet shows up. So instead the whole charging dance above happens immediately when a packet is received, and the data is copied later on. For quite a long time, I've thought it would be nifty if there was a NIC that kept received data in its own RAM and then allowed it to be efficiently DMAed to application memory when the application was ready for it. In essence, a lot of the accounting and memory management logic could move out of the kernel into the NIC. I'm not aware of anyone doing this.

fragmede 2y ago
RDMA is common for high performance applications but it doesn't work over the Internet.
- Danieru 2y ago
  It's a good thing the NIC is connected over pcie then.
- 2y ago

apitman 2y ago

Ditching head of line blocking is potentially a big win, but I really wish it wouldn't have come with so much complexity.

raggi 2y ago

UDP offload gets you implicitly today: - 64 packets per syscall, which is enough data to amortize the syscall overhead - a single packet is not. - UDP offload optionally lets you defer checksum computation, often offloading it to hardware. - UDP offload lets you skip/reuse route lookups for subsequent packets in a bundle. What UDP offload is no good for though, is large scale servers - the current APIs only work when the incoming packet chains neatly organize into batches per peer socket. If you have many thousands of active sockets you’ll stop having full bundles and the overhead starts sneaking back in. As I said in another thread, we really need a replacement for the BSD APIs here, they just don’t scale for modern hardware constraints and software needs - much too expensive per packet.

10000truths 2y ago

Bulk throughout isn't on par with TLS mainly because NICs with dedicated hardware for QUIC offload aren't commercially available (yet). Latency is undoubtedly better - the 1-RTT QUIC handshake substantially reduces time-to-first-byte compared to TLS.

majke 2y ago

> What is UDP offload going to do? Handling ACK packets in kernelspace would be one thing - helping for example RTT estimation. With userspace stack ACK's are handled in application and are subject to scheduler, suffering a lot on a loaded system.

morning-coffee 2y ago
There are no ACKs inherent in the UDP protocol, so "UDP offload" is not where the savings are. There are ACKs in the QUIC protocol and they are carried by UDP datagrams which need to make their way up to user land to be processed, and this is the crux of the issue. What is needed is for QUIC offload to be invented/supported by HW so that most of the high-frequency/tiny-packet processing happens there, just as it does today for TCP offload. TCP large-send and large-receive offload is what is responsible for all the CPU savings as the application deals in 64KB or larger send/receives and the segmentation and receive coalescing all happen in hardware before an interrupt is even generated to involve the kernel, let alone userland.