5 ms·

> Yes, uring is fancy, but there’s a tutorial level API middle ground possible that should be safe and 10x less overhead without resorting to uring level comple

> Yes, uring is fancy, but there’s a tutorial level API middle ground possible that should be safe and 10x less overhead without resorting to uring level complexity. I don't think io_uring is as complex as its reputation suggests. I don't think we need a substantially simpler low-level API; I think we need more high-level APIs built on top of io_uring. (That will also help with portability: we need APIs that can be most efficiently implemented atop io_uring but that work on non-Linux systems.)

raggi 2y ago
> I don't think io_uring is as complex as its reputation suggests. uring is extremely problematic to integrate into many common application / language runtimes and it has been demonstrably difficult to integrate into linux safely and correctly as well, with a continual stream of bugs, security and policy control issues. in principle a shared memory queue is a reasonable basis for improving the IO cost between applications and IO stacks such as the network or filesystem stacks, but this isn't easy to do well, cf. uring bugs and binder bugs.
- JoshTriplett 2y ago
  > with a continual stream of bugs, security and policy control issues This has not been true for a long time. There was an early design mistake that made it quite prone to these, but that mistake has been fixed. Unfortunately, the reputational damage will stick around for a while.
  - raggi 2y ago
    13 CVEs so far this year afaik
    - bonzini 2y ago
      CVE numbers from the Linux CNA are bollocks.
      - JoshTriplett 2y ago
        This conversation would be a good one to point them to to show that their policy is not just harmless point-proving, but in fact does cause harm. For context, to the best of my knowledge the current approach of the Linux CNA is, in keeping with long-standing Linux security policy of "every single fix might be a security fix", to assign CVEs regardless of whether something has any security impact or not.
        di4na 2y ago
        I would not call it harm. The use of uring in higher level languages is definitely prone to errors, bugs and security problems
        JoshTriplett 2y ago
        See the context I added to that comment; this is not about security issues, it's about the Linux CNA's absurd approach to CVE assignment for things that aren't CVEs.
        tialaramex 2y ago
        I don't agree that it's absurd. I would say it reflects a proper understanding of their situation. You've doubtless heard Tony Hoare's "There are two ways to write code: write code so simple there are obviously no bugs in it, or write code so complex that there are no obvious bugs in it.". Linux is definitely in the latter category, it's now such a sprawling system that determining whether a bug "really" has security implications is no long a reasonable task compared to just fixing the bug. The other reason is that Linux is so widely used that almost no assumption made to simplify that above task is definitely correct.
        JoshTriplett 2y ago
        That's fine, except that it is thus no longer meaningful to compare CVE count.
- jeffparsons 2y ago
  I find this surprising, given that my initial response to reading the iouring design was: 1. This is pretty clean and straightforward. 2. This is obviously what we need to decouple a bunch of things without the previous downsides. What has made it so hard to integrate it into common language runtimes? Do you have examples of where there's been an irreconcilable "impedance mismatch"?
  - raggi 2y ago
    https://github.com/tailscale/tailscale/pull/2370 https://github.com/tailscale/tailscale/pull/2370 was a practical drive toward this, will not proceed on this path. much more approachable, boats has written about challenges integrating in rust: https://without.boats/tags/io-uring/ https://without.boats/tags/io-uring/ in the most general form: you need a fairly "loose" memory model to integrate the "best" (performance wise) parts, and the "best" (ease of use/forward looking safety) way to integrate requires C library linkage. This is troublesome in most GC languages, and many managed runtimes. There's also the issue that uring being non-portable means that the things it suggests you must do (such as say pinning a buffer pool and making APIs like read not immediate caller allocates) requires a substantially separate API for this platform than for others, or at least substantial reworks over all the existing POSIX modeled APIs - thus back to what I said originally, we need a replacement for POSIX & BSD here, broadly applied.
    - gpderetta 2y ago
      I can see how a zero-copy API would be hard to implement on some languages, but you could still implement something on top of io_uring with posix buffer copy semantics , while using batching to decrease syscall overhead. Zero-copy APIs will necessarily be tricky to implement and use, especially on memory safe languages.
      - gmokki 2y ago
        I think most GC languages support native/pinned me(at least Java and C# do memory to support talking to kernel or native libraries. The APIs are even quite nice.
- arghwhat 2y ago
  Two things: One, uring is not extremely problematic to integrate, as it can be chained into a conventional event loop if you want to, or can even be fit into a conventionally blocking design to get localized syscall benefits. That is, you do not need to convert to a fully uring event loop design, even if that would be superior - and it can usually be kept entirely within a (slightly modified) event loop abstraction. The reason it has not yet been implemented is just priority - most stuff isn't bottlenecked on IOPS. Two, yes you could have e middle-ground. I assume the syscall overhead you call out is the need to send UDP packets one at a time through sendmsg/sendto, rather than doing one big write for several packets worth of data on TCP. An API that allowed you to provide a chain of messages, like sendmsg takes an iovec for data, is possible. But it's also possible to do this already as a tiny blocking wrapper around io_uring, saving you new syscalls.
  - Veserv 2y ago
    The system call to send multiple UDP packets in a single call has existed since Linux 3.0 over a decade ago[1]: sendmmsg(). [1] https://man7.org/linux/man-pages/man2/sendmmsg.2.html https://man7.org/linux/man-pages/man2/sendmmsg.2.html
    - arghwhat 2y ago
      Ah nice, in that case OP's point about syscall overhead is entirely moot. :) That should really be in the `SEE ALSO` of `man 3 sendmsg`...
      - evntdrvn 2y ago
        patches welcome :p
      - wtarreau 2y ago
        There's still the problem of sending to multiple destinations: OK sendmmsg() can send multiple datagrams, but for a given socket. When you have small windows (thank you cubic), you'll just send a few datagrams this way and don't save much.
lukeh 2y ago
async/await io_uwring wrappers for languages such as Swift [1] and Rust [2] [3] can improve usability considerably. I'm not super familiar with the Rust wrappers but, I've been using IORingSwift for socket, file and serial I/O for some time now. [1] https://github.com/PADL/IORingSwift https://github.com/PADL/IORingSwift [2] https://github.com/bytedance/monoio https://github.com/bytedance/monoio [3] https://github.com/tokio-rs/tokio-uring https://github.com/tokio-rs/tokio-uring
anarazel 2y ago
FWIW, the biggest problem I've seen with efficiently using io_uring for networking is that none of the popular TLS libraries have a buffer ownership model that really is suitable for asynchronous network IO. What you'd want is the ability to control the buffer for the "raw network side", so that asynchronous network IO can be performed without having to copy between a raw network buffer and buffers owned by the TLS library. It also would really help if TLS libraries supported processing multiple TLS records in a batched fashion. Doing roundtrips between app <-> tls library <-> userspace network buffer <-> kernel <-> HW for every 16kB isn't exactly efficient.

raggi 2y ago

> I don't think io_uring is as complex as its reputation suggests. uring is extremely problematic to integrate into many common application / language runtimes and it has been demonstrably difficult to integrate into linux safely and correctly as well, with a continual stream of bugs, security and policy control issues. in principle a shared memory queue is a reasonable basis for improving the IO cost between applications and IO stacks such as the network or filesystem stacks, but this isn't easy to do well, cf. uring bugs and binder bugs.

JoshTriplett 2y ago
> with a continual stream of bugs, security and policy control issues This has not been true for a long time. There was an early design mistake that made it quite prone to these, but that mistake has been fixed. Unfortunately, the reputational damage will stick around for a while.
- raggi 2y ago
  13 CVEs so far this year afaik
  - bonzini 2y ago
    CVE numbers from the Linux CNA are bollocks.
    - JoshTriplett 2y ago
      This conversation would be a good one to point them to to show that their policy is not just harmless point-proving, but in fact does cause harm. For context, to the best of my knowledge the current approach of the Linux CNA is, in keeping with long-standing Linux security policy of "every single fix might be a security fix", to assign CVEs regardless of whether something has any security impact or not.
      - di4na 2y ago
        I would not call it harm. The use of uring in higher level languages is definitely prone to errors, bugs and security problems
        JoshTriplett 2y ago
        See the context I added to that comment; this is not about security issues, it's about the Linux CNA's absurd approach to CVE assignment for things that aren't CVEs.
        tialaramex 2y ago
        I don't agree that it's absurd. I would say it reflects a proper understanding of their situation. You've doubtless heard Tony Hoare's "There are two ways to write code: write code so simple there are obviously no bugs in it, or write code so complex that there are no obvious bugs in it.". Linux is definitely in the latter category, it's now such a sprawling system that determining whether a bug "really" has security implications is no long a reasonable task compared to just fixing the bug. The other reason is that Linux is so widely used that almost no assumption made to simplify that above task is definitely correct.
        JoshTriplett 2y ago
        That's fine, except that it is thus no longer meaningful to compare CVE count.
jeffparsons 2y ago
I find this surprising, given that my initial response to reading the iouring design was: 1. This is pretty clean and straightforward. 2. This is obviously what we need to decouple a bunch of things without the previous downsides. What has made it so hard to integrate it into common language runtimes? Do you have examples of where there's been an irreconcilable "impedance mismatch"?
- raggi 2y ago
  https://github.com/tailscale/tailscale/pull/2370 https://github.com/tailscale/tailscale/pull/2370 was a practical drive toward this, will not proceed on this path. much more approachable, boats has written about challenges integrating in rust: https://without.boats/tags/io-uring/ https://without.boats/tags/io-uring/ in the most general form: you need a fairly "loose" memory model to integrate the "best" (performance wise) parts, and the "best" (ease of use/forward looking safety) way to integrate requires C library linkage. This is troublesome in most GC languages, and many managed runtimes. There's also the issue that uring being non-portable means that the things it suggests you must do (such as say pinning a buffer pool and making APIs like read not immediate caller allocates) requires a substantially separate API for this platform than for others, or at least substantial reworks over all the existing POSIX modeled APIs - thus back to what I said originally, we need a replacement for POSIX & BSD here, broadly applied.
  - gpderetta 2y ago
    I can see how a zero-copy API would be hard to implement on some languages, but you could still implement something on top of io_uring with posix buffer copy semantics , while using batching to decrease syscall overhead. Zero-copy APIs will necessarily be tricky to implement and use, especially on memory safe languages.
    - gmokki 2y ago
      I think most GC languages support native/pinned me(at least Java and C# do memory to support talking to kernel or native libraries. The APIs are even quite nice.
arghwhat 2y ago
Two things: One, uring is not extremely problematic to integrate, as it can be chained into a conventional event loop if you want to, or can even be fit into a conventionally blocking design to get localized syscall benefits. That is, you do not need to convert to a fully uring event loop design, even if that would be superior - and it can usually be kept entirely within a (slightly modified) event loop abstraction. The reason it has not yet been implemented is just priority - most stuff isn't bottlenecked on IOPS. Two, yes you could have e middle-ground. I assume the syscall overhead you call out is the need to send UDP packets one at a time through sendmsg/sendto, rather than doing one big write for several packets worth of data on TCP. An API that allowed you to provide a chain of messages, like sendmsg takes an iovec for data, is possible. But it's also possible to do this already as a tiny blocking wrapper around io_uring, saving you new syscalls.
- Veserv 2y ago
  The system call to send multiple UDP packets in a single call has existed since Linux 3.0 over a decade ago[1]: sendmmsg(). [1] https://man7.org/linux/man-pages/man2/sendmmsg.2.html https://man7.org/linux/man-pages/man2/sendmmsg.2.html
  - arghwhat 2y ago
    Ah nice, in that case OP's point about syscall overhead is entirely moot. :) That should really be in the `SEE ALSO` of `man 3 sendmsg`...
    - evntdrvn 2y ago
      patches welcome :p
    - wtarreau 2y ago
      There's still the problem of sending to multiple destinations: OK sendmmsg() can send multiple datagrams, but for a given socket. When you have small windows (thank you cubic), you'll just send a few datagrams this way and don't save much.

lukeh 2y ago

async/await io_uwring wrappers for languages such as Swift [1] and Rust [2] [3] can improve usability considerably. I'm not super familiar with the Rust wrappers but, I've been using IORingSwift for socket, file and serial I/O for some time now. [1] https://github.com/PADL/IORingSwift https://github.com/PADL/IORingSwift [2] https://github.com/bytedance/monoio https://github.com/bytedance/monoio [3] https://github.com/tokio-rs/tokio-uring https://github.com/tokio-rs/tokio-uring

anarazel 2y ago

FWIW, the biggest problem I've seen with efficiently using io_uring for networking is that none of the popular TLS libraries have a buffer ownership model that really is suitable for asynchronous network IO. What you'd want is the ability to control the buffer for the "raw network side", so that asynchronous network IO can be performed without having to copy between a raw network buffer and buffers owned by the TLS library. It also would really help if TLS libraries supported processing multiple TLS records in a batched fashion. Doing roundtrips between app <-> tls library <-> userspace network buffer <-> kernel <-> HW for every 16kB isn't exactly efficient.