5 ms·

I feel a bit paralyzed by Fear Of Missing Io_Uring. There's so much awesome streaming stuff about (RisingWave, Materialize, NATS, DataFusion, Velox, neat upstar

I feel a bit paralyzed by Fear Of Missing Io_Uring. There's so much awesome streaming stuff about (RisingWave, Materialize, NATS, DataFusion, Velox, neat upstarts like Iggy, many more), but it all feels built on slower legacy system libraries. It's not heavily used yet, but Rust has a bunch of fairly high visibility efforts. Situation sort of feels similar with http3, where the problem is figuring out what to pick. https://github.com/tokio-rs/tokio-uring https://github.com/tokio-rs/tokio-uring https://github.com/bytedance/monoio https://github.com/bytedance/monoio https://github.com/DataDog/glommio https://github.com/DataDog/glommio Alas libuv (powering Node.js) shipped io_uring but disabled it latter. Seems to have significantly worn out the original author on the topic to boot. https://github.com/libuv/libuv/pull/4421#issuecomment-2225860128 https://github.com/libuv/libuv/pull/4421#issuecomment-222586...

hnav 2y ago
io_uring is a low level abstraction and is generally a wash against epoll. Really won't make a difference for these kinds of applications, especially not for client nodes.
- 10000truths 2y ago
  io_uring allows for async reads and writes to disk without forcing a thread pool or direct I/O. That alone makes it much more scalable for workloads that touch both the network and disk.
  - immibis 2y ago
    Doesn't the kernel use a thread pool to process the requests in the ring, because the kernel is still designed around blocking disk I/O?
    - JoshTriplett 2y ago
      No, most operations in the ring directly work asynchronously. The thread mechanism only exists as a fallback for combinations of operations and system configurations (e.g. filesystems) that don't support asynchronous operation.
      - mightyham 2y ago
        I don't know anything about the internals of io_uring and am genuinely curious how it works. Saying it "directly works asynchronously" doesn't mean anything though. When circular buffer requests are processed what thread is processing the request, how is that thread managed, and how does it manage blocking/unblocking when communicating with the storage device?
        JoshTriplett 2y ago
        Internally, many parts of the Linux kernel operate asynchronously: they queue up a request with some subsystem (e.g. a hardware device), and get an event delivered when the request is completed. In such cases, io_uring can enqueue such a request, and complete it when receiving the event, without needing to use a thread to block waiting for it. See, for instance, https://lpc.events/event/11/contributions/901/attachments/786/1661/io_uring-BPF.pdf https://lpc.events/event/11/contributions/901/attachments/78... slide 5 (though more has happened since then). io_uring will first see if it has everything needed to do the operation immediately, if not it'll queue a request in some cases (e.g. direct I/O, or buffered I/O in some cases). The thread pool is the last fallback, which always works if nothing else does. https://lwn.net/Articles/821274/ https://lwn.net/Articles/821274/ talks about making async buffered reads work, for instance.
        haberman 2y ago
        Is it safe to say that a single thread using io_uring should be as fast or faster than N threads performing the same set of I/O tasks in a blocking manner? In other words, can you count on the kernel to use its own threads internally whenever an I/O task might actually need to use a lot of CPU?
        10000truths 2y ago
        If you saturate the submission queue with CPU-bottlenecked tasks, it defeats the value-add of io_uring - at that point, you might as well replace your kernel-space thread pool with a user-space one.
  - hnav 2y ago
    The point was that io_uring isn't going to make a big difference for the network code, as for disk I/O code (especially for the sorts of things GP is talking about) you have a bounded number of "threads" of execution anyway. For a node in a pub-sub system, maybe it has c10k users but it's probably appending to a handful of LSM-like datastructures that are written sequentially to disk. The biggest difference is random reads, but even then you can saturate what the disk will do with double digit numbers of threads.
immibis 2y ago
I seem to be missing context for this reply. Why do you need io_uring?
- jauntywundrkind 2y ago
  You don't need io_uring. For many workloads being slow & inefficient is acceptable, isn't awful. But gee I'd rather start from a modern baseline that has high levels of mechanistic sympathy with the hardware, where things like network & io work can be done in an efficient async manner. Why do I need io_uring? Because it sounds awful and unhackerly to suffer living in a much lesser worse world.
  - amluto 2y ago
    Mechanical sympathy is understanding the system, not using the shiniest thing. If you want low latency processing of one event at a time, you are either going to burn an entire core spinning or you are going to do a syscall for each operation. The io_uring syscalls are not especially fast — they get their awesomeness by doing, potentially, a whole lot of work per operation. And, for some use cases, by having a superior async IO model. But if you actually just want read(), then call read().
    - jauntywundrkind 2y ago
      Low latency for a single event is never going to have mechanistic sympathy, will be a colossal waste of most of your system. Highly concurrent system usage is what it takes. EPOLLEXCLUSIVE (2016) finally sort of gets epoll vaguely capable of what OSes were doing decades ago but is still difficult to use & a rats nest of complexity. Who here feels good reading https://stackoverflow.com/questions/41582560/how-does-epolls-epollexclusive-mode-interact-with-level-triggering https://stackoverflow.com/questions/41582560/how-does-epolls... ? The submission/completion queue model of io_uring makes sense. It lets work be added or resolved without crossing that painful slow kernel barrier. It's been expanded to offer a lot more operations than what could be done in epoll. The "shiniest thing" is a vast leap in capabilities, systems legibility, and overall (not single operation) throughout. You cannot remotely get the numbers io_uring was bringing three years ago any other way. And it's only gotten further and further ahead while everyone else has sat still.

hnav 2y ago

io_uring is a low level abstraction and is generally a wash against epoll. Really won't make a difference for these kinds of applications, especially not for client nodes.

10000truths 2y ago
io_uring allows for async reads and writes to disk without forcing a thread pool or direct I/O. That alone makes it much more scalable for workloads that touch both the network and disk.
- immibis 2y ago
  Doesn't the kernel use a thread pool to process the requests in the ring, because the kernel is still designed around blocking disk I/O?
  - JoshTriplett 2y ago
    No, most operations in the ring directly work asynchronously. The thread mechanism only exists as a fallback for combinations of operations and system configurations (e.g. filesystems) that don't support asynchronous operation.
    - mightyham 2y ago
      I don't know anything about the internals of io_uring and am genuinely curious how it works. Saying it "directly works asynchronously" doesn't mean anything though. When circular buffer requests are processed what thread is processing the request, how is that thread managed, and how does it manage blocking/unblocking when communicating with the storage device?
      - JoshTriplett 2y ago
        Internally, many parts of the Linux kernel operate asynchronously: they queue up a request with some subsystem (e.g. a hardware device), and get an event delivered when the request is completed. In such cases, io_uring can enqueue such a request, and complete it when receiving the event, without needing to use a thread to block waiting for it. See, for instance, https://lpc.events/event/11/contributions/901/attachments/786/1661/io_uring-BPF.pdf https://lpc.events/event/11/contributions/901/attachments/78... slide 5 (though more has happened since then). io_uring will first see if it has everything needed to do the operation immediately, if not it'll queue a request in some cases (e.g. direct I/O, or buffered I/O in some cases). The thread pool is the last fallback, which always works if nothing else does. https://lwn.net/Articles/821274/ https://lwn.net/Articles/821274/ talks about making async buffered reads work, for instance.
        haberman 2y ago
        Is it safe to say that a single thread using io_uring should be as fast or faster than N threads performing the same set of I/O tasks in a blocking manner? In other words, can you count on the kernel to use its own threads internally whenever an I/O task might actually need to use a lot of CPU?
        10000truths 2y ago
        If you saturate the submission queue with CPU-bottlenecked tasks, it defeats the value-add of io_uring - at that point, you might as well replace your kernel-space thread pool with a user-space one.
- hnav 2y ago
  The point was that io_uring isn't going to make a big difference for the network code, as for disk I/O code (especially for the sorts of things GP is talking about) you have a bounded number of "threads" of execution anyway. For a node in a pub-sub system, maybe it has c10k users but it's probably appending to a handful of LSM-like datastructures that are written sequentially to disk. The biggest difference is random reads, but even then you can saturate what the disk will do with double digit numbers of threads.

immibis 2y ago

I seem to be missing context for this reply. Why do you need io_uring?

jauntywundrkind 2y ago
You don't need io_uring. For many workloads being slow & inefficient is acceptable, isn't awful. But gee I'd rather start from a modern baseline that has high levels of mechanistic sympathy with the hardware, where things like network & io work can be done in an efficient async manner. Why do I need io_uring? Because it sounds awful and unhackerly to suffer living in a much lesser worse world.
- amluto 2y ago
  Mechanical sympathy is understanding the system, not using the shiniest thing. If you want low latency processing of one event at a time, you are either going to burn an entire core spinning or you are going to do a syscall for each operation. The io_uring syscalls are not especially fast — they get their awesomeness by doing, potentially, a whole lot of work per operation. And, for some use cases, by having a superior async IO model. But if you actually just want read(), then call read().
  - jauntywundrkind 2y ago
    Low latency for a single event is never going to have mechanistic sympathy, will be a colossal waste of most of your system. Highly concurrent system usage is what it takes. EPOLLEXCLUSIVE (2016) finally sort of gets epoll vaguely capable of what OSes were doing decades ago but is still difficult to use & a rats nest of complexity. Who here feels good reading https://stackoverflow.com/questions/41582560/how-does-epolls-epollexclusive-mode-interact-with-level-triggering https://stackoverflow.com/questions/41582560/how-does-epolls... ? The submission/completion queue model of io_uring makes sense. It lets work be added or resolved without crossing that painful slow kernel barrier. It's been expanded to offer a lot more operations than what could be done in epoll. The "shiniest thing" is a vast leap in capabilities, systems legibility, and overall (not single operation) throughout. You cannot remotely get the numbers io_uring was bringing three years ago any other way. And it's only gotten further and further ahead while everyone else has sat still.