4 ms·

Two things: One, uring is not extremely problematic to integrate, as it can be chained into a conventional event loop if you want to, or can even be fit into a

Two things: One, uring is not extremely problematic to integrate, as it can be chained into a conventional event loop if you want to, or can even be fit into a conventionally blocking design to get localized syscall benefits. That is, you do not need to convert to a fully uring event loop design, even if that would be superior - and it can usually be kept entirely within a (slightly modified) event loop abstraction. The reason it has not yet been implemented is just priority - most stuff isn't bottlenecked on IOPS. Two, yes you could have e middle-ground. I assume the syscall overhead you call out is the need to send UDP packets one at a time through sendmsg/sendto, rather than doing one big write for several packets worth of data on TCP. An API that allowed you to provide a chain of messages, like sendmsg takes an iovec for data, is possible. But it's also possible to do this already as a tiny blocking wrapper around io_uring, saving you new syscalls.

Veserv 2y ago
The system call to send multiple UDP packets in a single call has existed since Linux 3.0 over a decade ago[1]: sendmmsg(). [1] https://man7.org/linux/man-pages/man2/sendmmsg.2.html https://man7.org/linux/man-pages/man2/sendmmsg.2.html
- arghwhat 2y ago
  Ah nice, in that case OP's point about syscall overhead is entirely moot. :) That should really be in the `SEE ALSO` of `man 3 sendmsg`...
  - evntdrvn 2y ago
    patches welcome :p
  - wtarreau 2y ago
    There's still the problem of sending to multiple destinations: OK sendmmsg() can send multiple datagrams, but for a given socket. When you have small windows (thank you cubic), you'll just send a few datagrams this way and don't save much.
    - arghwhat 2y ago
      > There's still the problem of sending to multiple destinations: OK sendmmsg() can send multiple datagrams, but for a given socket. Hmm? sendmsg takes the destination address in the `struct msghdr` structure, and sendmmsg takes an array of those structures. At the same time, the discussion of efficiency is about UDP vs. TCP. TCP writes are per socket, to the connected peer, and so UDP has the upper hand here. The concerns were about how TCP allows giving a large buffer to the kernel in a single write that then gets sliced into smaller packets automatically, vs. having to slice it in userspace and call send more, which sendmmsg solves. (You can of course do single-syscall or even zero-syscall "send to many" with io_uring for any socket type, but that's a different discussion.)
      - wtarreau 2y ago
        > > There's still the problem of sending to multiple destinations: OK sendmmsg() can send multiple datagrams, but for a given socket. > Hmm? sendmsg takes the destination address in the `struct msghdr` structure, and sendmmsg takes an array of those structures. But that's still pointless on a connected socket. And if you're not using connected sockets, you're performing destination lookups for each and every datagram you're trying to send. It also means you're running with small buffers by default (the 212kB default buffers per socket are shared with all your destinations, no longer per destination). Thus normally you want to use connected socket when dealing with UDP in environments having performance requirements.
- justincormack 2y ago
  At one point if I remember it didnt actually work, it still just sent one message at a time and returned the length of the first piece of the iovec. Hopefully it got fixed.
- johnp_ 2y ago
  Looks like Mozilla is currently working on implementing `sendmmsg` and `recvmmsg` use in neqo (Mozilla's QUIC implementation) [1]. [1] https://github.com/mozilla/neqo/issues/1693 https://github.com/mozilla/neqo/issues/1693
londons_explore 2y ago
I think you need to look at a common use case and consider how many syscalls you'd like it to take and how many CPU cycles would be reasonable. Let's take downloading a 1MB jpeg image over QUIC and rendering it on the screen. I would hope that can be done in about 100k CPU cycles and 20 syscalls, considering that all the jpeg decoding and rendering is going to be hardware accelerated. The decryption is also hardware accelerated. Unfortunately, no network API allows that right now. The CPU needs to do a substantial amount of processing for every individual packet, in both userspace and kernel space, for receiving the packet and sending the ACK, and there is no 'bulk decrypt' non-blocking API. Even the data path is troublesome - there should be a way for the data to go straight from the network card to the GPU, with the CPU not even touching it, but we're far from that.
- arghwhat 2y ago
  There's a few issues here. 1. A 1 MB file is at the very least 64 individually encrypted TLS records (16k max size) sent in sequence, possibly more. So decryption 64 times is the maximum amount of bulk work you can do - this is done to allow streaming verification and decryption in parallel with the download, whereas one big block would have you wait for the very last byte before any processing could start. 2. TLS is still userspace and decryption does not involve the kernel, and thus no syscalls. The benefits of kernel TLS largely focus on servers sending files straight from disk, bypassing userspace for the entire data processing path. This is not really relevant receive-side for something you are actively decoding. 3. JPEG is, to my knowledge, rarely hardware offloaded on desktop, so no syscalls there. Now, the number of actual syscalls end up being dictated by the speed of the sender, and the tunable receive buffer size. The slower the sender, the more kernel roundtrips you end upo with, which allows you to amortize the processing over a longer period so everything is ready when the last packet is. For a fast enough sender with big enough receive buffers, this could be a single kernel roundtrip.
  - miohtama 2y ago
    JPEG is not a particular great example. However most video streams and partially hardware decoded. Usually you still need to decode part of the stream, namely entropy coding and metadata, first on the CPU.
immibis 2y ago
This system call you're asking for already exists - it's called sendmmsg. There is also recvmmsg.

Veserv 2y ago

The system call to send multiple UDP packets in a single call has existed since Linux 3.0 over a decade ago[1]: sendmmsg(). [1] https://man7.org/linux/man-pages/man2/sendmmsg.2.html https://man7.org/linux/man-pages/man2/sendmmsg.2.html

arghwhat 2y ago
Ah nice, in that case OP's point about syscall overhead is entirely moot. :) That should really be in the `SEE ALSO` of `man 3 sendmsg`...
- evntdrvn 2y ago
  patches welcome :p
- wtarreau 2y ago
  There's still the problem of sending to multiple destinations: OK sendmmsg() can send multiple datagrams, but for a given socket. When you have small windows (thank you cubic), you'll just send a few datagrams this way and don't save much.
  - arghwhat 2y ago
    > There's still the problem of sending to multiple destinations: OK sendmmsg() can send multiple datagrams, but for a given socket. Hmm? sendmsg takes the destination address in the `struct msghdr` structure, and sendmmsg takes an array of those structures. At the same time, the discussion of efficiency is about UDP vs. TCP. TCP writes are per socket, to the connected peer, and so UDP has the upper hand here. The concerns were about how TCP allows giving a large buffer to the kernel in a single write that then gets sliced into smaller packets automatically, vs. having to slice it in userspace and call send more, which sendmmsg solves. (You can of course do single-syscall or even zero-syscall "send to many" with io_uring for any socket type, but that's a different discussion.)
    - wtarreau 2y ago
      > > There's still the problem of sending to multiple destinations: OK sendmmsg() can send multiple datagrams, but for a given socket. > Hmm? sendmsg takes the destination address in the `struct msghdr` structure, and sendmmsg takes an array of those structures. But that's still pointless on a connected socket. And if you're not using connected sockets, you're performing destination lookups for each and every datagram you're trying to send. It also means you're running with small buffers by default (the 212kB default buffers per socket are shared with all your destinations, no longer per destination). Thus normally you want to use connected socket when dealing with UDP in environments having performance requirements.
justincormack 2y ago
At one point if I remember it didnt actually work, it still just sent one message at a time and returned the length of the first piece of the iovec. Hopefully it got fixed.
johnp_ 2y ago
Looks like Mozilla is currently working on implementing `sendmmsg` and `recvmmsg` use in neqo (Mozilla's QUIC implementation) [1]. [1] https://github.com/mozilla/neqo/issues/1693 https://github.com/mozilla/neqo/issues/1693

londons_explore 2y ago

I think you need to look at a common use case and consider how many syscalls you'd like it to take and how many CPU cycles would be reasonable. Let's take downloading a 1MB jpeg image over QUIC and rendering it on the screen. I would hope that can be done in about 100k CPU cycles and 20 syscalls, considering that all the jpeg decoding and rendering is going to be hardware accelerated. The decryption is also hardware accelerated. Unfortunately, no network API allows that right now. The CPU needs to do a substantial amount of processing for every individual packet, in both userspace and kernel space, for receiving the packet and sending the ACK, and there is no 'bulk decrypt' non-blocking API. Even the data path is troublesome - there should be a way for the data to go straight from the network card to the GPU, with the CPU not even touching it, but we're far from that.

arghwhat 2y ago
There's a few issues here. 1. A 1 MB file is at the very least 64 individually encrypted TLS records (16k max size) sent in sequence, possibly more. So decryption 64 times is the maximum amount of bulk work you can do - this is done to allow streaming verification and decryption in parallel with the download, whereas one big block would have you wait for the very last byte before any processing could start. 2. TLS is still userspace and decryption does not involve the kernel, and thus no syscalls. The benefits of kernel TLS largely focus on servers sending files straight from disk, bypassing userspace for the entire data processing path. This is not really relevant receive-side for something you are actively decoding. 3. JPEG is, to my knowledge, rarely hardware offloaded on desktop, so no syscalls there. Now, the number of actual syscalls end up being dictated by the speed of the sender, and the tunable receive buffer size. The slower the sender, the more kernel roundtrips you end upo with, which allows you to amortize the processing over a longer period so everything is ready when the last packet is. For a fast enough sender with big enough receive buffers, this could be a single kernel roundtrip.
- miohtama 2y ago
  JPEG is not a particular great example. However most video streams and partially hardware decoded. Usually you still need to decode part of the stream, namely entropy coding and metadata, first on the CPU.

immibis 2y ago

This system call you're asking for already exists - it's called sendmmsg. There is also recvmmsg.