13 ms·

Efficient IO with io_uring [pdf]

255 points by diegocg 7y ago

aseipp 7y ago
For a bit of advertisement and any interested parties: I packaged liburing in the NixOS unstable branch a week or two ago (Jens hasn't done a stable release just yet, however), and I also pushed the 5.1 kernel release into unstable last night. Hurrah for a new, efficient I/O interface finally appearing!
zbjornson 7y ago
I made a prototype of a replacement for Node.js' fs.read/write using io_uring in hopes of it ultimately replacing libuv's current threadpool implementation. Definitely promising! Many props to Jens for his hard work on this and for his helpful discussions on the API. Benchmarks at [0] and prototype repo at [1] [0] https://github.com/libuv/libuv/issues/1947#issuecomment-485230126 https://github.com/libuv/libuv/issues/1947#issuecomment-4852... [1] https://github.com/zbjornson/node-io-uring https://github.com/zbjornson/node-io-uring
- coder543 7y ago
  io_uring sounds promising. It'll be interesting to see how long it takes for Tokio to add support for it in the Rust community, and how long it takes for Go to transparently support this instead of a pool of file I/O workers.
  - jcranmer 7y ago
    The bug is here, if you want to follow it: https://github.com/tokio-rs/mio/issues/923 https://github.com/tokio-rs/mio/issues/923
  - Matthias247 7y ago
    It will be interesting to see whether Rusts async ecosystem gets support for it. However there is a little bit of a challenge in the form of an impedance mismatch: Rust Futures and thereby also async functions require that an operation is cancellable at any point of time (by dropping the respective Future). IO completion based systems (like io_uring and IOCP) typically don't support this. If the operation in the Rust userspace would be cancelled while the Kernel proceeds with the operation the Kernel would act on invalid memory. There are some workarounds for this, like wrapping the APIs in ones that only used owned buffers that are passed for the whole duration of the operation and reference-counting the IO objects (file descriptors). But the behavior is pretty tricky. [1] describes the problem. [1] https://github.com/rust-lang-nursery/futures-rs/issues/1278 https://github.com/rust-lang-nursery/futures-rs/issues/1278
    - coder543 7y ago
      That's really interesting. I think some kind of reference counting and a shadow event loop is going to be necessary, if that's the case, so that I/O futures can always be driven to completion even if the developer-facing one is dropped.
      - Matthias247 7y ago
        Yes, reference-counting of outstanding operations and the used buffers is a suitable workaround. It comes with some downsides though too: The IO operations then only accept certain specialized buffer types and nothing generic like pure byte slices anymore, which might make those things less interoperable. The other issue is that while using refcounting leaves at least the memory in a non-corrupted state the IO source will be in a somewhat undefined state after the cancellation. E.g. if a write operation had been issued on the socket, the API user can not make any assumptions whether or not any byte had been written, and which impact a subsequent write might have. The only reasonable thing to do is to close the IO source (where the close might also need to wait until the still pending background task has completed).
        coder543 7y ago
        If the future has been dropped, I agree that the source/sink should be closed as soon as safely possible. Usually the resource has to be moved into a future for it to be used, and thus it's logical that no one could continue using it after the future is dropped.
    - GolDDranks 7y ago
      I wonder if it would be possible to pass the ownership of a structure that contains the buffer to kernel, passing a pointer in the user_data. The application could then `std::mem::forget` the buffer, and re-construct it from the raw pointer when the completion event arrives. The Future that originated the IO job could then be dropped independently from the buffer.
    - the8472 7y ago
      io_uring definitely wants to own the buffers, either for its entire lifetime (with preregistered memory mappings) or at least until completion. if possible the future should take ownership of the buffer and have it returned on completion.
  - stbuehler 7y ago
    https://github.com/stbuehler/rust-io-uring https://github.com/stbuehler/rust-io-uring Note that io_uring in linux 5.1 has various bugs; especially READ* and WRITE* on sockets/pipe don't work very well (afaik they either block submission or return -EAGAIN with O_NONBLOCK), so the example code in my repo isn't that useful for linux 5.1 (POLL_ADD works fine though). For proper support in tokio this would have to be done in mio afaict (or tokio would have to replace mio with something else).
    - coder543 7y ago
      do those issues only apply to networking? the big benefit I see for io_uring is that it provides a truly async interface for file I/O for the first time ever in Linux... as far as I can tell.
      - stbuehler 7y ago
        I'm not sure about the details for file I/O; files with FMODE_NOWAIT should be fine (block devices, ext4, btrfs, ...). The idea is that during submission an operation is started with IOCB_NOWAIT; an operation shouldn't block with that. If it returns EAGAIN it will be retried in another kernel thread without IOCB_NOWAIT. Afaik IOCB_NOWAIT support should be announced through FMODE_NOWAIT. Right now operations that don't support IOCB_NOWAIT ("magic check" in io_file_supports_async [1]) are still run during submission but without the IOCB_NOWAIT flag. Sockets/pipes don't handle IOCB_NOWAIT, they only care about O_NONBLOCK right now (which is a bug imho). [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/io_uring.c?h=v5.1#n823 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
- emilsedgh 7y ago
  That's a huge improvement! Really looking forward to Node taking advantage of this.
- mmastrac 7y ago
  If I'm reading your chart correctly, it looks like the thread pool of size 1 is still "beating" the uring, right? What would explain this?
  - BeeOnRope 7y ago
    Lower numbers are better in that chart, so the threadpool size one is only winning on the rightmost part of the plat: just barely at parallelism 16 and slightly more at 32.
    - mmastrac 7y ago
      Makes sense - thanks. Looks like this is a huge win for low parallelism system and a very minor loss at high parallelism.
PaulHoule 7y ago
How many times have we heard this story that there is a new async I/O interface for Linux that is better than the others? How is this one different?
- rapsey 7y ago
  Being usable for writing to disk is pretty huge. Solves so many problems for webservers and databases.
  - deleted 7y ago
    [deleted]
- strmpnk 7y ago
  The linked PDF does a very good job answering that and thankfully the first couple paragraphs should give a good start to understanding how and what it solves. The specific point to make is that while this can work with socket handles, it's primarily aimed to allow more efficient work with files (where AIO has failed to provide much benefit outside of batching operations with fewer syscalls). io_uring is also one of the first post-spectre/meltdown syscall designs to be merged into the Linux kernel. If the ring buffer style used in this API becomes a pattern or not will be interesting.
- xxpor 7y ago
  Using ring buffers is how networking has worked for a while now. They're nice because they're extremely fast, due to being in mechanical sympathy with hardware. http://mechanitis.blogspot.com/2011/06/dissecting-disruptor-whats-so-special.html http://mechanitis.blogspot.com/2011/06/dissecting-disruptor-... So I can't really speak to the API details, but the general concept seems like a good idea to me.
  - kazinator 7y ago
    For values of "a while" probably exceeding forty years.
    - 7y ago
waynesonfire 7y ago
Can anyone comment on how this interface compares to state-of-the-art async i/o in other *unix o/s? Is Linux playing catch up here or innovating?
- crest 7y ago
  Looks like a little bit of both. Linux has a lot to catch up because epoll is a mess. It is limited to file descriptors with a readiness semantic (sockets, pipes, ttys) meaning that it will never work with normal files because they are always ready while blocking whenever they feel like it. The second API fuck up is the fork/exec semantic. Oh and they tried to get away with edge triggered instead of level triggered. All in all epoll is a just a nasty case of NIH. The epoll developers just wrote code without understanding the prior art in their field (e.g. kqueue).
  - AnanasAttack 7y ago
    What do you dislike about edge triggered mode?
  - oralty 7y ago
    epoll (and even kqueue) is not aio and was never intended as such. While epoll had some early snafus, those have long been fixed (offering ET and LT) and it works well for its intended purpose (io readiness). This (io_uring) is more for a thing like Windows overlapped IO and "unix" (incl FreeBSD) aio. AIO on linux is yet a separate set of syscalls (io_*) and has since day one in the 2.5 days been pretty much a useless joke, very limited in scope (O_DIRECT only). kqueue is neither epoll nor aio but a common queueing interface for events, which includes io readiness and aio completion as separate types. Linux prefers to split these as different types of fds. FWIW, this wasn't due to obliviousness of the existing design - https://yarchive.net/comp/linux/event_queues.html https://yarchive.net/comp/linux/event_queues.html. Honestly your comment sounds very like a regurgitated old and somewhat outdated Bryan Cantrill rant.
    - gpderetta 7y ago
      The link doesn't explain much. The only argument that Linus has on kqueue is that is overengineered. Then it goes on a long multi message rant in how having multiple queues is insane. Luckily, in the end epoll ended up supporting multiple queues at least. I do believe that ET is good though.
PyroLagus 7y ago
Interestingly, Genode actually also uses two rings for async bulk transfer RPC (packet streams)[0], only the rings are called submit queue and acknowledgement queue rather than submission queue and completion queue. And since Genode applications do I/O by communicating with a service implementing the block session interface[1], which uses the aforementioned packet streams, Genode essentially uses two ring buffers for async I/O just like Linux does (well, I guess it's more the other way around). I can definitely recommend reading through the first half of Genode Foundations (the second half is just an API reference). It's really interesting and informative, and it's a surprisingly easy read as well. However, I'd wait till the 19.05 edition gets released, since that should be less than a month's wait if they manage to release it on time. [0] https://genode.org/documentation/genode-foundations-18-05.pdf https://genode.org/documentation/genode-foundations-18-05.pd... (page 92) [1] (pages 94 & 116)
- salotz 7y ago
  Was about to say the same thing! +1 for reading the Genode manual. It is the best resource I have ever found for understanding operating systems, security, and architecture. The discussion of multiplexers and report-ROM modules is pertinent here as well (pg 133).
gruez 7y ago
How many async io APIs are we up to now?
- kazinator 7y ago
  Give me your mobile number and go do something else, I will send you a notification with the answer. If you run out of something else to do, then just wait.
- jeltz 7y ago
  Just two actually (io_submit and io_uring), unless we are also counting network IO where we also have select/poll/epoll.
kazinator 7y ago
I did something very similar for a logging system, more than ten years ago. Circular buffers were shared between user space and kernel. (Allocated by user space, mapped into kernel with get_user_pages.) User space threads just added messages to their circular buffer by copying in data, hitting a barrier and updating a head pointer. Like io_uring, I avoided too many system calls by having a kernel thread poll for changes. User space would only make a system call when the ring would near a "high water mark", like 85% full or whatever. The circular buffers could outlive the process which created them and attached them. A process could crash and go away; the kernel thread would still process the last messages written right up to the crash. Not everyone around was keen on the idea. What do you mean just stick it into a buffer and the kernel will pick it up. Polling? Bah.
- strictfp 7y ago
  Interesting implementation. How did the buffers survive process crash? Is that a side-effect of get_user_pages or did you have a userspace daemon?
  - kazinator 7y ago
    It was a side effect of get_user_pages, which obtains durable references to pages that have to be explicitly dropped. These were held by the logging module independently of the processes interacting with it. (The kernel thread was actually provide by a system call from user space. So there was a daemon as a user space program, but that just called into the kernel, where it then sat in a loop. This is nice: you have something in user space that you can kill with SIGTERM or whatever and restart by running some /usr/bin/executable.) If the daemon terminated, that didn't destroy any of the buffers; they would just sit there getting backlogged. New ones could be created, too (via ioctl calls on a character device in /dev). One aspect was that get_user_pages doesn't give you a linear block of memory. I wrote the grotty C code to dequeue messages from the non-linearly-mapped circular buffer. The message headers could straddle page boundaries, so I copied those out into temporaries before working with them. I think there was some attempt to avoid making unnecessary copies of the bulk data.
ignoramous 7y ago
The problems identified for aio on Linux are: > (parts snipped for brevity) > The biggest limitation is undoubtedly that it only supports async IO for O_DIRECT (or un-buffered) accesses. Due to the restrictions of O_DIRECT (cache bypassing and size/alignment restraints), this makes the native aio interface a no-go for most use cases. > There are a number of ways async-io submission can end up blocking - if meta data is required to perform IO, the submission will block waiting for that. For storage devices, there are a fixed number of request slots available. If those slots are currently all in use, submission will block waiting for one to become available. > The API isn't great. Each IO submission ends up needing to copy 64 + 8 bytes and each completion copies 32 bytes. That's 104 bytes of memory copy. IO always requires at least two system calls (submit + wait-for-completion), which in these post spectre/meltdown days is a serious slowdown. The io_uring soln: > With a shared ring buffer, we could eliminate the need to have shared locking between the application and the kernel, getting away with some clever use of memory ordering and barriers instead. There are two fundamental operations associated with an async interface: the act of submitting a request, and the event that is associated with the completion of said request. For submitting IO, the application is the producer and the kernel is the consumer. The opposite is true for completions - here the kernel produces completion events and the application consumes them. Hence, we need a pair of rings to provide an effective communication channel between an application and the kernel. That pair of rings is at the core of the new interface, io_uring. They are suitably named submission queue (SQ), and completion queue (CQ), and form the foundation of the new interface. > The cqes are organized into an array, with the memory backing the array being visible and modifiable by both the kernel and the application. However, since the cqe's are produced by the kernel, only the kernel is actually modifying the cqe entries. The communication is managed by a ring buffer. Whenever a new event is posted by the kernel to the CQ ring, it updates the tail associated with it. When the application consumes an entry, it updates the head. Hence, if the tail is different than the head, the application knows that it has one or more events available for consumption. The ring counters themselves are free flowing 32-bit integers, and rely on natural wrapping when the number of completed events exceed the capacity of the ring. > For the submission side, the roles are reversed. The application is the one updating the tail, and the kernel consumes entries (and updates) the head. One important difference is that while the CQ ring is directly indexing the shared array of cqes, the submission side has an indirection array between them. Hence the submission side ring buffer is an index into this array, which in turn contains the index into the sqes. This might initially seem odd and confusing, but there's some reasoning behind it. Some applications may embed request units inside internal data structures, and this allows them the flexibility to do so while retaining the ability to submit multiple sqes in one. I've seen the ring-buffer design pattern employed here by io_uring in log4j via disruptor [0] and the queue structure in libfabric interface for Infiniband [1] for both command/control and data. Happy to see Jens Axboe [2] committing his time to improve story around aio. [0] https://www.baeldung.com/lmax-disruptor-concurrency https://www.baeldung.com/lmax-disruptor-concurrency [1] https://zcopy.wordpress.com/2010/10/08/quick-concepts-part-1-%e2%80%93-introduction-to-rdma/ https://zcopy.wordpress.com/2010/10/08/quick-concepts-part-1... [2] https://en.wikipedia.org/wiki/Jens_Axboe https://en.wikipedia.org/wiki/Jens_Axboe
Const-me 7y ago
I like the design. On paper, it's even more efficient than Windows registered I/O. However, one thing, in 8.3 > This is done through a kernel thread, speciﬁc to that io_uring. Does it means it will be 100k native kernel threads for an app that has 100k open sockets? Won't this ruin scalability?
- Matthias247 7y ago
  It's one thread per ring, not one thread per socket or FileDescriptor that dispatches events to the ring.
  - Const-me 7y ago
    How many rings should I create on a server with 100GB RAM, 64 cores, that handles 100k connections? Even if there's a definite answer to that, I still don't understand why threads are necessary? While CPU waits for completed IO, it doesn't poll anything. The CPU fires a DMA request at hardware, and forgets about it until it gets the response.
    - geocar 7y ago
      It depends on your application, so measure! Ideally the CPU never waits for completed IO, it's busy building the next set of responses, or writing out the new state to the rest of the application. If you're ever "waiting", then you should probably be putting another/bigger nic in there or rethinking what your application actually needs to do: 10Gb ethernet is only about 10x slower than main memory, so 10 threads could be enough if you have a single memory fetch to service the response, and if you need two, then 20 threads. If you need more than six, then you should definitely rearchitect your application, since you're out of cores and time-slicing again.
      - cafxx 7y ago
        > 10Gb ethernet is only about 10x slower than main memory Server-class CPUs have much higher bandwidth nowadays: > The total memory bandwidth available to a single CPU clocks in at 170 GB/s. (i.e. ~136x slower than main memory) https://www.anandtech.com/show/11183/amd-prepares-32-core-naples-cpus-for-1p-and-2p-servers-coming-in-q2 https://www.anandtech.com/show/11183/amd-prepares-32-core-na...
dmoreno 7y ago
As explained by LWN: https://lwn.net/Articles/776703/ https://lwn.net/Articles/776703/
jorangreef 7y ago
Does FreeBSD have anything like io_uring?
- Const-me 7y ago
  As far as I know, it does not have similar functionality. But I’m not sure it needs one. It has kqueue, and unlike epoll, kqueue is a general-purpose way to get events from kernel. It’s compatible not just with sockets, but also files and pipes, directory changes, processes, signals, timers, devices, and more. Not unlike IOCP on Windows. On paper new uring is faster, but kqueue is not particularly slow either. And it’s available for couple decades now, i.e. it’s stable and tested really well. I’m happy Linux finally getting fast general purpose async IO, but I’ll only benefit from it after a year or so, after kernel updates are out, and after higher-level runtimes like golang and .net core are updated accordingly.
  - machinecoffee 7y ago
    There's also libkqueue for Linux - I would imagine they will also adopt io_uring when it's stable and ubiquitous enough, bringing the two APIs closer together. I find kqueue to be a great and well designed API, so that can only be a good thing IMHO.
deleted 7y ago
[deleted]

255 points by diegocg 7y ago

For a bit of advertisement and any interested parties: I packaged liburing in the NixOS unstable branch a week or two ago (Jens hasn't done a stable release just yet, however), and I also pushed the 5.1 kernel release into unstable last night. Hurrah for a new, efficient I/O interface finally appearing!

zbjornson 7y ago

I made a prototype of a replacement for Node.js' fs.read/write using io_uring in hopes of it ultimately replacing libuv's current threadpool implementation. Definitely promising! Many props to Jens for his hard work on this and for his helpful discussions on the API. Benchmarks at [0] and prototype repo at [1] [0] https://github.com/libuv/libuv/issues/1947#issuecomment-485230126 https://github.com/libuv/libuv/issues/1947#issuecomment-4852... [1] https://github.com/zbjornson/node-io-uring https://github.com/zbjornson/node-io-uring

coder543 7y ago
io_uring sounds promising. It'll be interesting to see how long it takes for Tokio to add support for it in the Rust community, and how long it takes for Go to transparently support this instead of a pool of file I/O workers.
- jcranmer 7y ago
  The bug is here, if you want to follow it: https://github.com/tokio-rs/mio/issues/923 https://github.com/tokio-rs/mio/issues/923
- Matthias247 7y ago
  It will be interesting to see whether Rusts async ecosystem gets support for it. However there is a little bit of a challenge in the form of an impedance mismatch: Rust Futures and thereby also async functions require that an operation is cancellable at any point of time (by dropping the respective Future). IO completion based systems (like io_uring and IOCP) typically don't support this. If the operation in the Rust userspace would be cancelled while the Kernel proceeds with the operation the Kernel would act on invalid memory. There are some workarounds for this, like wrapping the APIs in ones that only used owned buffers that are passed for the whole duration of the operation and reference-counting the IO objects (file descriptors). But the behavior is pretty tricky. [1] describes the problem. [1] https://github.com/rust-lang-nursery/futures-rs/issues/1278 https://github.com/rust-lang-nursery/futures-rs/issues/1278
  - coder543 7y ago
    That's really interesting. I think some kind of reference counting and a shadow event loop is going to be necessary, if that's the case, so that I/O futures can always be driven to completion even if the developer-facing one is dropped.
    - Matthias247 7y ago
      Yes, reference-counting of outstanding operations and the used buffers is a suitable workaround. It comes with some downsides though too: The IO operations then only accept certain specialized buffer types and nothing generic like pure byte slices anymore, which might make those things less interoperable. The other issue is that while using refcounting leaves at least the memory in a non-corrupted state the IO source will be in a somewhat undefined state after the cancellation. E.g. if a write operation had been issued on the socket, the API user can not make any assumptions whether or not any byte had been written, and which impact a subsequent write might have. The only reasonable thing to do is to close the IO source (where the close might also need to wait until the still pending background task has completed).
      - coder543 7y ago
        If the future has been dropped, I agree that the source/sink should be closed as soon as safely possible. Usually the resource has to be moved into a future for it to be used, and thus it's logical that no one could continue using it after the future is dropped.
  - GolDDranks 7y ago
    I wonder if it would be possible to pass the ownership of a structure that contains the buffer to kernel, passing a pointer in the user_data. The application could then `std::mem::forget` the buffer, and re-construct it from the raw pointer when the completion event arrives. The Future that originated the IO job could then be dropped independently from the buffer.
  - the8472 7y ago
    io_uring definitely wants to own the buffers, either for its entire lifetime (with preregistered memory mappings) or at least until completion. if possible the future should take ownership of the buffer and have it returned on completion.
- stbuehler 7y ago
  https://github.com/stbuehler/rust-io-uring https://github.com/stbuehler/rust-io-uring Note that io_uring in linux 5.1 has various bugs; especially READ* and WRITE* on sockets/pipe don't work very well (afaik they either block submission or return -EAGAIN with O_NONBLOCK), so the example code in my repo isn't that useful for linux 5.1 (POLL_ADD works fine though). For proper support in tokio this would have to be done in mio afaict (or tokio would have to replace mio with something else).
  - coder543 7y ago
    do those issues only apply to networking? the big benefit I see for io_uring is that it provides a truly async interface for file I/O for the first time ever in Linux... as far as I can tell.
    - stbuehler 7y ago
      I'm not sure about the details for file I/O; files with FMODE_NOWAIT should be fine (block devices, ext4, btrfs, ...). The idea is that during submission an operation is started with IOCB_NOWAIT; an operation shouldn't block with that. If it returns EAGAIN it will be retried in another kernel thread without IOCB_NOWAIT. Afaik IOCB_NOWAIT support should be announced through FMODE_NOWAIT. Right now operations that don't support IOCB_NOWAIT ("magic check" in io_file_supports_async [1]) are still run during submission but without the IOCB_NOWAIT flag. Sockets/pipes don't handle IOCB_NOWAIT, they only care about O_NONBLOCK right now (which is a bug imho). [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/io_uring.c?h=v5.1#n823 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
emilsedgh 7y ago
That's a huge improvement! Really looking forward to Node taking advantage of this.
mmastrac 7y ago
If I'm reading your chart correctly, it looks like the thread pool of size 1 is still "beating" the uring, right? What would explain this?
- BeeOnRope 7y ago
  Lower numbers are better in that chart, so the threadpool size one is only winning on the rightmost part of the plat: just barely at parallelism 16 and slightly more at 32.
  - mmastrac 7y ago
    Makes sense - thanks. Looks like this is a huge win for low parallelism system and a very minor loss at high parallelism.

PaulHoule 7y ago

How many times have we heard this story that there is a new async I/O interface for Linux that is better than the others? How is this one different?

rapsey 7y ago
Being usable for writing to disk is pretty huge. Solves so many problems for webservers and databases.
- deleted 7y ago
  [deleted]
strmpnk 7y ago
The linked PDF does a very good job answering that and thankfully the first couple paragraphs should give a good start to understanding how and what it solves. The specific point to make is that while this can work with socket handles, it's primarily aimed to allow more efficient work with files (where AIO has failed to provide much benefit outside of batching operations with fewer syscalls). io_uring is also one of the first post-spectre/meltdown syscall designs to be merged into the Linux kernel. If the ring buffer style used in this API becomes a pattern or not will be interesting.
xxpor 7y ago
Using ring buffers is how networking has worked for a while now. They're nice because they're extremely fast, due to being in mechanical sympathy with hardware. http://mechanitis.blogspot.com/2011/06/dissecting-disruptor-whats-so-special.html http://mechanitis.blogspot.com/2011/06/dissecting-disruptor-... So I can't really speak to the API details, but the general concept seems like a good idea to me.
- kazinator 7y ago
  For values of "a while" probably exceeding forty years.
  - 7y ago

waynesonfire 7y ago

Can anyone comment on how this interface compares to state-of-the-art async i/o in other *unix o/s? Is Linux playing catch up here or innovating?

crest 7y ago
Looks like a little bit of both. Linux has a lot to catch up because epoll is a mess. It is limited to file descriptors with a readiness semantic (sockets, pipes, ttys) meaning that it will never work with normal files because they are always ready while blocking whenever they feel like it. The second API fuck up is the fork/exec semantic. Oh and they tried to get away with edge triggered instead of level triggered. All in all epoll is a just a nasty case of NIH. The epoll developers just wrote code without understanding the prior art in their field (e.g. kqueue).
- AnanasAttack 7y ago
  What do you dislike about edge triggered mode?
- oralty 7y ago
  epoll (and even kqueue) is not aio and was never intended as such. While epoll had some early snafus, those have long been fixed (offering ET and LT) and it works well for its intended purpose (io readiness). This (io_uring) is more for a thing like Windows overlapped IO and "unix" (incl FreeBSD) aio. AIO on linux is yet a separate set of syscalls (io_*) and has since day one in the 2.5 days been pretty much a useless joke, very limited in scope (O_DIRECT only). kqueue is neither epoll nor aio but a common queueing interface for events, which includes io readiness and aio completion as separate types. Linux prefers to split these as different types of fds. FWIW, this wasn't due to obliviousness of the existing design - https://yarchive.net/comp/linux/event_queues.html https://yarchive.net/comp/linux/event_queues.html. Honestly your comment sounds very like a regurgitated old and somewhat outdated Bryan Cantrill rant.
  - gpderetta 7y ago
    The link doesn't explain much. The only argument that Linus has on kqueue is that is overengineered. Then it goes on a long multi message rant in how having multiple queues is insane. Luckily, in the end epoll ended up supporting multiple queues at least. I do believe that ET is good though.

PyroLagus 7y ago

Interestingly, Genode actually also uses two rings for async bulk transfer RPC (packet streams)[0], only the rings are called submit queue and acknowledgement queue rather than submission queue and completion queue. And since Genode applications do I/O by communicating with a service implementing the block session interface[1], which uses the aforementioned packet streams, Genode essentially uses two ring buffers for async I/O just like Linux does (well, I guess it's more the other way around). I can definitely recommend reading through the first half of Genode Foundations (the second half is just an API reference). It's really interesting and informative, and it's a surprisingly easy read as well. However, I'd wait till the 19.05 edition gets released, since that should be less than a month's wait if they manage to release it on time. [0] https://genode.org/documentation/genode-foundations-18-05.pdf https://genode.org/documentation/genode-foundations-18-05.pd... (page 92) [1] (pages 94 & 116)

salotz 7y ago
Was about to say the same thing! +1 for reading the Genode manual. It is the best resource I have ever found for understanding operating systems, security, and architecture. The discussion of multiplexers and report-ROM modules is pertinent here as well (pg 133).

gruez 7y ago

How many async io APIs are we up to now?

kazinator 7y ago
Give me your mobile number and go do something else, I will send you a notification with the answer. If you run out of something else to do, then just wait.
jeltz 7y ago
Just two actually (io_submit and io_uring), unless we are also counting network IO where we also have select/poll/epoll.

kazinator 7y ago

I did something very similar for a logging system, more than ten years ago. Circular buffers were shared between user space and kernel. (Allocated by user space, mapped into kernel with get_user_pages.) User space threads just added messages to their circular buffer by copying in data, hitting a barrier and updating a head pointer. Like io_uring, I avoided too many system calls by having a kernel thread poll for changes. User space would only make a system call when the ring would near a "high water mark", like 85% full or whatever. The circular buffers could outlive the process which created them and attached them. A process could crash and go away; the kernel thread would still process the last messages written right up to the crash. Not everyone around was keen on the idea. What do you mean just stick it into a buffer and the kernel will pick it up. Polling? Bah.

strictfp 7y ago
Interesting implementation. How did the buffers survive process crash? Is that a side-effect of get_user_pages or did you have a userspace daemon?
- kazinator 7y ago
  It was a side effect of get_user_pages, which obtains durable references to pages that have to be explicitly dropped. These were held by the logging module independently of the processes interacting with it. (The kernel thread was actually provide by a system call from user space. So there was a daemon as a user space program, but that just called into the kernel, where it then sat in a loop. This is nice: you have something in user space that you can kill with SIGTERM or whatever and restart by running some /usr/bin/executable.) If the daemon terminated, that didn't destroy any of the buffers; they would just sit there getting backlogged. New ones could be created, too (via ioctl calls on a character device in /dev). One aspect was that get_user_pages doesn't give you a linear block of memory. I wrote the grotty C code to dequeue messages from the non-linearly-mapped circular buffer. The message headers could straddle page boundaries, so I copied those out into temporaries before working with them. I think there was some attempt to avoid making unnecessary copies of the bulk data.

ignoramous 7y ago

The problems identified for aio on Linux are: > (parts snipped for brevity) > The biggest limitation is undoubtedly that it only supports async IO for O_DIRECT (or un-buffered) accesses. Due to the restrictions of O_DIRECT (cache bypassing and size/alignment restraints), this makes the native aio interface a no-go for most use cases. > There are a number of ways async-io submission can end up blocking - if meta data is required to perform IO, the submission will block waiting for that. For storage devices, there are a fixed number of request slots available. If those slots are currently all in use, submission will block waiting for one to become available. > The API isn't great. Each IO submission ends up needing to copy 64 + 8 bytes and each completion copies 32 bytes. That's 104 bytes of memory copy. IO always requires at least two system calls (submit + wait-for-completion), which in these post spectre/meltdown days is a serious slowdown. The io_uring soln: > With a shared ring buffer, we could eliminate the need to have shared locking between the application and the kernel, getting away with some clever use of memory ordering and barriers instead. There are two fundamental operations associated with an async interface: the act of submitting a request, and the event that is associated with the completion of said request. For submitting IO, the application is the producer and the kernel is the consumer. The opposite is true for completions - here the kernel produces completion events and the application consumes them. Hence, we need a pair of rings to provide an effective communication channel between an application and the kernel. That pair of rings is at the core of the new interface, io_uring. They are suitably named submission queue (SQ), and completion queue (CQ), and form the foundation of the new interface. > The cqes are organized into an array, with the memory backing the array being visible and modifiable by both the kernel and the application. However, since the cqe's are produced by the kernel, only the kernel is actually modifying the cqe entries. The communication is managed by a ring buffer. Whenever a new event is posted by the kernel to the CQ ring, it updates the tail associated with it. When the application consumes an entry, it updates the head. Hence, if the tail is different than the head, the application knows that it has one or more events available for consumption. The ring counters themselves are free flowing 32-bit integers, and rely on natural wrapping when the number of completed events exceed the capacity of the ring. > For the submission side, the roles are reversed. The application is the one updating the tail, and the kernel consumes entries (and updates) the head. One important difference is that while the CQ ring is directly indexing the shared array of cqes, the submission side has an indirection array between them. Hence the submission side ring buffer is an index into this array, which in turn contains the index into the sqes. This might initially seem odd and confusing, but there's some reasoning behind it. Some applications may embed request units inside internal data structures, and this allows them the flexibility to do so while retaining the ability to submit multiple sqes in one. I've seen the ring-buffer design pattern employed here by io_uring in log4j via disruptor [0] and the queue structure in libfabric interface for Infiniband [1] for both command/control and data. Happy to see Jens Axboe [2] committing his time to improve story around aio. [0] https://www.baeldung.com/lmax-disruptor-concurrency https://www.baeldung.com/lmax-disruptor-concurrency [1] https://zcopy.wordpress.com/2010/10/08/quick-concepts-part-1-%e2%80%93-introduction-to-rdma/ https://zcopy.wordpress.com/2010/10/08/quick-concepts-part-1... [2] https://en.wikipedia.org/wiki/Jens_Axboe https://en.wikipedia.org/wiki/Jens_Axboe

Const-me 7y ago

I like the design. On paper, it's even more efficient than Windows registered I/O. However, one thing, in 8.3 > This is done through a kernel thread, speciﬁc to that io_uring. Does it means it will be 100k native kernel threads for an app that has 100k open sockets? Won't this ruin scalability?

Matthias247 7y ago
It's one thread per ring, not one thread per socket or FileDescriptor that dispatches events to the ring.
- Const-me 7y ago
  How many rings should I create on a server with 100GB RAM, 64 cores, that handles 100k connections? Even if there's a definite answer to that, I still don't understand why threads are necessary? While CPU waits for completed IO, it doesn't poll anything. The CPU fires a DMA request at hardware, and forgets about it until it gets the response.
  - geocar 7y ago
    It depends on your application, so measure! Ideally the CPU never waits for completed IO, it's busy building the next set of responses, or writing out the new state to the rest of the application. If you're ever "waiting", then you should probably be putting another/bigger nic in there or rethinking what your application actually needs to do: 10Gb ethernet is only about 10x slower than main memory, so 10 threads could be enough if you have a single memory fetch to service the response, and if you need two, then 20 threads. If you need more than six, then you should definitely rearchitect your application, since you're out of cores and time-slicing again.
    - cafxx 7y ago
      > 10Gb ethernet is only about 10x slower than main memory Server-class CPUs have much higher bandwidth nowadays: > The total memory bandwidth available to a single CPU clocks in at 170 GB/s. (i.e. ~136x slower than main memory) https://www.anandtech.com/show/11183/amd-prepares-32-core-naples-cpus-for-1p-and-2p-servers-coming-in-q2 https://www.anandtech.com/show/11183/amd-prepares-32-core-na...

dmoreno 7y ago

As explained by LWN: https://lwn.net/Articles/776703/ https://lwn.net/Articles/776703/

jorangreef 7y ago

Does FreeBSD have anything like io_uring?

Const-me 7y ago
As far as I know, it does not have similar functionality. But I’m not sure it needs one. It has kqueue, and unlike epoll, kqueue is a general-purpose way to get events from kernel. It’s compatible not just with sockets, but also files and pipes, directory changes, processes, signals, timers, devices, and more. Not unlike IOCP on Windows. On paper new uring is faster, but kqueue is not particularly slow either. And it’s available for couple decades now, i.e. it’s stable and tested really well. I’m happy Linux finally getting fast general purpose async IO, but I’ll only benefit from it after a year or so, after kernel updates are out, and after higher-level runtimes like golang and .net core are updated accordingly.
- machinecoffee 7y ago
  There's also libkqueue for Linux - I would imagine they will also adopt io_uring when it's stable and ubiquitous enough, bringing the two APIs closer together. I find kqueue to be a great and well designed API, so that can only be a good thing IMHO.

deleted 7y ago

[deleted]