13 ms·

Io_uring is not an event system

281 points by ot 5y ago

api 5y ago
What's funny about io_uring is that the blocking syscall interface was always something critics of Unix pointed out as being a major shortcoming. We had OSes that did this kind of thing way back in the 1980s and 1990s, but Unix with its simplicity and generalism and free implementations took over. Now Unix is finally, in 2021, getting a syscall queue construct where I can interact with the kernel asynchronously.
- CodesInChaos 5y ago
  io_uring is Linux specific. BSD offers kqueue instead, introduced in 2000. I believe both are limited to specific operations, mostly IO, and aren't fully general asynchronous syscall interfaces.
  - rapsey 5y ago
    The entire point of the article is that it is not just for IO.
    - binarycrusader 5y ago
      The OP said mostly IO not only.
    - CodesInChaos 5y ago
      > The entire point of the article is that it is not just for IO I went through the list of operations supported by `io_uring_enter`. Almost all of them are for IO, the remainder (NOP, timeout, madvise) are useful for supporting IO, though madvise might have some non-IO uses as well. While io_uring could form the basis of a generic async syscall interface in the future, in its current state it most certainly is not. The article mostly talks about io_uring enabling completion based IO instead of readiness based IO. AFAIK kqueue also supports completion based IO using aio_read/aio_write together with sigevent.
      - deleted 5y ago
        [deleted]
      - coder543 5y ago
        > AFAIK kqueue also supports completion based IO using aio_read/aio_write together with sigevent. If you can point to a practical example of a program doing it this way and seeing a performance benefit, I would be curious to see it. I did some googling and didn't really even find any articles mentioning this as possibility. kqueue is widely considered to be readiness based, just like epoll, not completion based. What you wrote sounds like an interesting hack, but I'm not sure it counts for much if it is impractical to use.
        CodesInChaos 5y ago
        I don't know if it offers any practical benefits for sequential/socket IO. But AFAIK it's the way to go if you want to do async random-access/file IO.
        funny_falcon 5y ago
        Afaik, Nginx used this hack for aio on FreeBSD many years. It is superrior to aio on Linux. That is why Netflix used nginx together with FreeBSD. Now nginx could use threads for file io, and linux file performance became ok as well.
        mzs 5y ago
        It feels like it's been ten years since I fiddled with this on FreeBSD but EVFILT_AIO worked surprisingly well with aio and kqueue on spinning disk.
        asveikau 5y ago
        POSIX AIO uses `struct sigevent` to notify completion. On FreeBSD you can set ->sigev_notify to SIGEV_KEVENT, and ->sigev_notify_kqueue to a kqueue descriptor. As for your "performance benefit" question and nobody using it over readiness based approaches: Well yes. No idea how it compares with a traditional readiness based socket program. But it's kind of not the point either. AFAIK the scenario where POSIX AIO makes most sense is files on disk.
  - lonjil 5y ago
    There's nothing stopping io_uring from gaining more non-IO operations, and indeed that appears to be the plan.
  - asveikau 5y ago
    kqueue is not really a replacement for io_uring. That said, FreeBSD can make a POSIX AIO request signal completion to a kqueue, which is a similar model of async I/O. But as has been pointed out elsewhere, io_uring is also about making other syscalls asynchronous.
- binarycrusader 5y ago
  Solaris had this over a decade ago now with event ports which are basically a variant of Windows IOCP: https://web.archive.org/web/20110719052845/http://developers.sun.com/solaris/articles/event_completion.html https://web.archive.org/web/20110719052845/http://developers... So at least one UNIX system had them a while ago.
  - wahern 5y ago
    People keep saying this but IME that's not how the Solaris Event Ports API works at all. The semantics of Solaris Event Ports is nearly identical to both epoll+family and kqueue. And like with both those others (ignoring io_uring), I/O completion is done using the POSIX AIO interface, which signals completion through the Event Port descriptor. I've written at least two wrapper libraries for I/O readiness, POSIX signal, file event, and user-triggered event polling that encompass epoll, kqueue, and Solaris Event Ports. Supporting all three is relatively trivial from an API perspective because they work so similarly. In fact, notably all three let you poll on the epoll, kqueue, or Event Port descriptor itself. So you can have event queue trees, which is very handy when writing composable libraries.
    - deleted 5y ago
      [deleted]
- PaulDavisThe1st 5y ago
  Maybe Linux will get scheduler activations in the near future, another OS feature from the 90s that ended up in Solaris and more or less nowhere else. "Let my user space thread scheduler do its work!"
  - aseipp 5y ago
    There's already plans for a new futex-based swap_to primitive, for improving userland thread scheduling capabilities. There was some work done on it last year, but it was rejected on LKML. At this rate, it looks like it will not move forward until the new futex2 syscall is in place, since the original API is showing its age. So, it will probably happen Soon™, but you're probably still ~2 years out before you can reliably depend on it, I'd say.
    - PaulDavisThe1st 5y ago
      Scheduler activations don't require swap_to. The kernel wakes up the user space scheduler when it decides to put the process onto a cpu. The user space scheduler decides which user space thread executes in the kernel thread context that it runs in, and does a user space thread switch (not a full context switch) to it. It's a combination of kernel threads and user space (aka "green") threads.
  - rektide 5y ago
    It might! I'm not sure if it's an exact fit or not but the User Managed Concurrency Groups work[1] Google is trying to upstream with their Fibers userland-scheduling library sounds like it could be a match, and perhaps it could get the upstreaming itcs seeking. [1] https://www.phoronix.com/scan.php?page=news_item&px=Google-Fibers-Toward-Open https://www.phoronix.com/scan.php?page=news_item&px=Google-F...
    - deleted 5y ago
      [deleted]
  - gpderetta 5y ago
    I think some of the *BSDs have (or had) it. Linux almost got it at the turn of the millennium, with the Next Generation Posix Threading project, but then the much simpler and faster NPTL won.
- ww520 5y ago
  I remember IPX has similar communication model. To read from the network, you post an array of buffer pointers to the IPX driver and can continue to do whatever. When the buffers are filled, the driver calls your completion function.
- mpweiher 5y ago
  Do you have some references?
joshmarinacci 5y ago
I'm curious how this handles the case where the calling program dies or wants to cancel the request before it's actually happened.
- asdfasgasdgasdg 5y ago
  I don't know the answer but I would assume if you have submitted an operation to the kernel, you should assume it's in an indeterminate state until you get the result. If the program dies then the call may complete or not. For cancellation there is an API. Example call: https://github.com/axboe/liburing/blob/c4c280f31b0e05a1ea792ce255dc8689aad1c887/test/io-cancel.c#L131 https://github.com/axboe/liburing/blob/c4c280f31b0e05a1ea792...
- PaulDavisThe1st 5y ago
  Death: same way it handles a program with an open socket and data arriving and unread and the program dies. It's just part of the overall resource set of the process and has to be cleaned up when the process goes away.
- ww520 5y ago
  The do_exit() function in kernel/exit.c is responsible for general cleanup on a process [1]. Whether a process dies gracefully or abruptly, the kernel calls do_exit() to clean up all the resources owned by the process, like opened files or acquired locks. I would imagine the io_uring related stuff is cleaned up there as well. [1] https://elixir.bootlin.com/linux/v5.13-rc6/source/kernel/exit.c https://elixir.bootlin.com/linux/v5.13-rc6/source/kernel/exi... Edit: I just looked up at the latest version of the source [1]. Yes, it does clean up io_uring related files.
vkka 5y ago
... So it has the potential to make a lot of programs much simpler. More efficient? Yes. Simpler? Not really. A synchronous program would be simpler, everyone who has done enough of these know it.
- masklinn 5y ago
  I’m guessing they mean that programs which did not want to block on syscalls and had to deploy workarounds can now just… do async syscalls.
  - Filligree 5y ago
    Which is every program that wants to be fast, on modern computers. So...
    - masklinn 5y ago
      Nonsense. Let's say your program wants to list a directory, if it has nothing to do during that time then there is no point to using an asynchronous model, that only adds costs and complexity.
      - wtallis 5y ago
        True. But as soon as your program wants to list two directories and knows both names ahead of time, you have an opportunity to fire off both operations for the kernel to work on simultaneously. And even if your program doesn't have any opportunity for doing IO in parallel, being able to chain a sequence of IO operations together and issue them with at most one syscall may still get you improved latency.
        deleted 5y ago
        [deleted]
        touisteur 5y ago
        Yes and even clustering often-used-together syscalls... An interesting 2010 thesis https://os.itec.kit.edu/deutsch/2211.php https://os.itec.kit.edu/deutsch/2211.php and https://www2.cs.arizona.edu/~debray/Publications/multi-call.pdf https://www2.cs.arizona.edu/~debray/Publications/multi-call.... for something called 'multi-calls' Interesting times.
tele_ski 5y ago
I think this might be the best explanation I've read of why io_uring should be better than epoll since it effectively collapses the 'tell me when this is ready' with the 'do action' part. That was the really enlightening part for me. I have to say though, the name io_uring seems unfortunate and I think the author touches on this in the article... the name is really an implementation detail but io_uring's true purpose is a generic asynchronous syscall facility that is currently tailored towards i/o. syscall_queue or async_queue or something else...? A descriptive api name and not an implementation detail would probably go a long way in helping the feature be easier to understand. Even window's IOCP seems infinitely better named than 'uring'.
- eloff 5y ago
  I've seen mixed results so far. In theory it should perform better than epoll, but I'm not sure it's quite there yet. The maintainer of uWebSockets tried it with an earlier version and it was slower. Where it really shines is disk IO because we don't have an epoll equivalent there. I imagine it would also be great at network requests that go to or from disk in a simple way because you can chain the syscalls in theory.
  - zxzax 5y ago
    The main benefit to me has been in cases that previously required a thread pool on top of epoll, it should be safe to get rid of the thread pool now and only use io_uring. Socket I/O of course doesn't really need a thread pool in a lot of cases, but disk I/O does.
  - eloff 5y ago
    I dug up the thread with the benchmarks: https://github.com/axboe/liburing/issues/189 https://github.com/axboe/liburing/issues/189 io_uring does win on more recent versions, but it's not like it blows epoll out of the water - it's an incremental improvement. Specifically for websockets where you have a lot of idle connections, you don't want a bunch of buffers waiting for reads for long periods of time - this is why Go doesn't perform well for websocket servers as calling Read() on a socket requires tying up both a buffer and a goroutine stack. I haven't looked into how registering buffers works with io_uring, if it's 1-to-1 mapping or if you can pass a pool of N buffers that can be used for reads on M sockets where M > N. The details matter for that specific case. Again where io_uring really shines is file IO because there are no good solutions there currently.
diegocg 5y ago
IIRC the author of io_uring wants to support using it with any system call, not just the current supported subset. Not sure how these efforts are going.
- swiley 5y ago
  Supporting exec sounds like it could be a security issue. I also have a hard time imagining why on earth you would call exec that way.
  - tptacek 5y ago
    Say more about this? You're probably right but I haven't thought about it before and I'd be happy to have you do that thinking for me. :)
    - touisteur 5y ago
      Start a user-provided compression background task at file close? Think tcpdump with io_uring 'take this here buffer you've just read and send it to disk without bothering userland.
  - the_duke 5y ago
    > Supporting exec sounds like it could be a security issue. Why would it be any more of an issue than calling a blocking exec? > why on earth you would call exec that way The same reason as why you would want to do anything else with io_uring? In an async runtime you have to delegate blocking calls to a thread pool. Much nicer if the runtime can use the same execution system as for other IO.
    - swiley 5y ago
      >Why would it be any more of an issue than calling a blocking exec? Now you turn everything doing fast I/O into something that's essentially an easier rwx situation WRT memory corruption. >Much nicer In other words: the runtime should handle it because the kernel doesn't need to.
rjzzleep 5y ago
I found this presentation to be quite informative https://www.youtube.com/watch?v=-5T4Cjw46ys https://www.youtube.com/watch?v=-5T4Cjw46ys
ayanamist 5y ago
So linux people find that the model of windows iocp used is better?
- hermanradtke 5y ago
  As a Linux person: Yes, and I have thought so for a long time. This is why many of us are excited about io_uring.
- ot 5y ago
  Yes, and now things are going full circle and windows is adopting the ring model too. https://windows-internals.com/i-o-rings-when-one-i-o-operation-is-not-enough/ https://windows-internals.com/i-o-rings-when-one-i-o-operati...
  - muststopmyths 5y ago
    huh, interesting. As far as I can tell the main advantage this has over IOCP is that you can get one completion for multiple read requests. Looks like they took a lot of the concepts from Winsock RIO and applied them to file I/O. Which is fascinating because with network traffic you can't predict packet boundaries and thus your I/O rate can be unpredictable. RIO helps you get the notification rate under control, which can help if your packet rate is very high. With files, I would think you can control the rate at which you request data, as well as the memory you allocate for it. The other thing it saves just like RIO is the overhead of locking/unlocking buffers by preregistering them. Is that the main reason for this API then ? I would be very interested to hear from people who have actually run into limits with overlapped file reads and are therefore excited about IoRings
- asveikau 5y ago
  I think it's long been understood that it's better for disk I/O. I'm not sure the consensus is as clear for sockets.
- volta83 5y ago
  Yes. Unfortunately Rust went the exact other way.
grok22 5y ago
Isn't locking a problem with io_uring? Won't you block the kernel when it's trying to do the event completion stuff and the completion work tries to take a lock? Or is the completion stuff done entirely in user-space and blocking is not a problem? Maybe I need to read up on this a bit more...
- zxzax 5y ago
  The submissions and completions are stored in a lock-free ring buffer, hence the name "uring."
  - legulere 5y ago
    lock-free ring buffers still have locks for the case when they are full, so it would be interesting to see how the kernel behaves when you never read from the completion ring.
    - zxzax 5y ago
      You will get an error for that upon submit. The application can then buffer the submissions somewhere else and wait for a few completions to finish.
    - jlokier 5y ago
      As long as command submission is only permitted (or considered valid) when the submitter ensures there is sufficient space in the completion ring for the replies, writing to the completion ring can't legitimately block, and the completion writer can assume this. This asymmetry is because the command submitter is able to block and is in control. It even works with elastic ringbuffers that can be expanded as needed. All memory allocation can be on the command submitter side. This trick works in other domains, not just kernels. For example request-reply ringbuffer pairs can be used to communicate with interrupt handlers, in OSes and also in bare metal embedded systems, and the interrupt handler does not need to block when writing to the reply ringbuffer. It's a nice way to queue things without locking issues on the interrupt side. Similarly with signal handlers and attached devices (e.g. NVMe storage, GPUs, network cards), those can can always write to the reply ringbuffer too.
ginsmar 5y ago
Very very interesting.
justsomeuser 5y ago
If my process has async/await and uses epoll, wouldn't this have about the same performance as io_uring? E.g. with io_uring, the event triggers the kernel to read into a buffer. With async/await epoll wakes up my process which does the syscall to read the file. In both cases you still need to read from the device and get the data to the user process?
- Misdicorl 5y ago
  Imagine your process instead of getting woken up to make a syscall gets woken up with a pointer to a filled data buffer.
  - justsomeuser 5y ago
    But some process (kernel or user space) still needs to spend the computers finite resources to read it? If the work happens in the kernel process or user process it still costs the same?
    - Misdicorl 5y ago
      Yes, the syscall work still happens and the raw work has not changed. The overhead has dropped dramatically though since you've eliminated at least 2 context switches per data fill cycle(and probably more).
    - wtallis 5y ago
      The transition from userspace to the kernel and back takes a similar amount of time to actually doing a small read from the fastest SSDs, or issuing a write to most relatively fast SSDs. So avoiding one extra syscall is a meaningful performance improvement even if you can't always eliminate a memcpy operation.
      - Veserv 5y ago
        That sounds unlikely. syscall hardware overhead (entry + exit) on a modern x86 is only on the order of 100 ns which is approximately main memory access latency. I am not familiar with Linux internals or what the fastest SSDs are capable of these days, but I am fairly sure that for your statement to be true Linux would need to be adding 1 to 2 orders of magnitude in software overhead. This occurs in the context switch pathway due to scheduling decisions, but it is fairly unlikely it occurs in the syscall pathway unless they are doing something horribly wrong.
mzs 5y ago
bandwidth exceeded alternative: https://web.archive.org/web/20210617150204/https://despairlabs.com/posts/2021-06-16-io-uring-is-not-an-event-system/ https://web.archive.org/web/20210617150204/https://despairla...
- zootboy 5y ago
  It seems that their server is rewriting the 451 error to a 403, which caused Archive.org to drop the page from its archives. Unfortunate...
bilalhusain 5y ago
Currently serving Bandwidth Restricted page - 451. Cached version of the write up https://archive.is/VgHkW https://archive.is/VgHkW
mikedilger 5y ago
> io_uring is not an event system at all. io_uring is actually a generic asynchronous syscall facility. I would call it a message passing system, not an asynchronous syscall facility. A syscall, even one that doesn't block indefinitely, transfers control to the kernel. io_uring, once setup, doesn't. Now that we have multiple cores, there is no reason to context switch if the kernel can handle your request on some other core on which it is perhaps already running.
- volta83 5y ago
  This is how I think about it as well. Which makes me wonder if Mach wasn't right all along.
  - mikedilger 5y ago
    I've been intending to try process-to-process message passing using a lock-free data structure very much like io_uring where the processes share pages (one page read-write, the other page readonly, and inverse for the other process) on hardware with L2 tightly-integrated memory (SiFive FU740). The result being zero-copy message passing at L2 cache speeds. [for an embedded realtime custom OS, not for linux]
- hinkley 5y ago
  It becomes a question of how expensive the message passing is between cores versus the context switch overhead - especially in a world where switching privilege levels has been made expensive by multiple categories of attack against processor speculation. Is there a middle ground where io_uring doesn't require the mitigations but other syscalls do?
raphlinus 5y ago
Something I've been thinking about that maybe the HN hivemind can help with; I know enough about io_uring and GPU each to be dangerous. The roundtrip for command buffer submission in GPU is huge by my estimation, around 100µs. On a 10TFLOPS card, which is nowhere near the top of the line, that's 1 billion operations. I don't know exactly where all the time is going, but suspect it's a bunch of process and kernel transitions between the application, the userland driver, and the kernel driver. My understanding is that games mostly work around this by batching up a lot of work (many dozens of draw calls, for example) in one submission. But it's still a problem if CPU readback is part of the workload. So my question is: can a technique like io_uring be used here, to keep the GPU pipeline full and only take expensive transitions when absolutely needed? I suspect the programming model will be different and in some cases harder, but that's already part of the territory with GPU.
- boardwaalk 5y ago
  The communication between the host and the GPU already works on a ring buffer on any modern GPU I believe. It’s why graphics APIs are asynchronous until you synchronize f.e. by flipping a frame buffer or reading something back. APIs like Vulkan are very explicit about this and have fences and semaphores. Older APIs will just block if you do something that requires blocking.
bullen 5y ago
To me this sounds like more kernel, I was hoping for less or no kernel instead. On my live machines the kernel is taking 30% CPU copying memory. I'm waiting for userspace networking... file IO can be offset with async-to-async parallelism.

281 points by ot 5y ago

api 5y ago

What's funny about io_uring is that the blocking syscall interface was always something critics of Unix pointed out as being a major shortcoming. We had OSes that did this kind of thing way back in the 1980s and 1990s, but Unix with its simplicity and generalism and free implementations took over. Now Unix is finally, in 2021, getting a syscall queue construct where I can interact with the kernel asynchronously.

CodesInChaos 5y ago
io_uring is Linux specific. BSD offers kqueue instead, introduced in 2000. I believe both are limited to specific operations, mostly IO, and aren't fully general asynchronous syscall interfaces.
- rapsey 5y ago
  The entire point of the article is that it is not just for IO.
  - binarycrusader 5y ago
    The OP said mostly IO not only.
  - CodesInChaos 5y ago
    > The entire point of the article is that it is not just for IO I went through the list of operations supported by `io_uring_enter`. Almost all of them are for IO, the remainder (NOP, timeout, madvise) are useful for supporting IO, though madvise might have some non-IO uses as well. While io_uring could form the basis of a generic async syscall interface in the future, in its current state it most certainly is not. The article mostly talks about io_uring enabling completion based IO instead of readiness based IO. AFAIK kqueue also supports completion based IO using aio_read/aio_write together with sigevent.
    - deleted 5y ago
      [deleted]
    - coder543 5y ago
      > AFAIK kqueue also supports completion based IO using aio_read/aio_write together with sigevent. If you can point to a practical example of a program doing it this way and seeing a performance benefit, I would be curious to see it. I did some googling and didn't really even find any articles mentioning this as possibility. kqueue is widely considered to be readiness based, just like epoll, not completion based. What you wrote sounds like an interesting hack, but I'm not sure it counts for much if it is impractical to use.
      - CodesInChaos 5y ago
        I don't know if it offers any practical benefits for sequential/socket IO. But AFAIK it's the way to go if you want to do async random-access/file IO.
      - funny_falcon 5y ago
        Afaik, Nginx used this hack for aio on FreeBSD many years. It is superrior to aio on Linux. That is why Netflix used nginx together with FreeBSD. Now nginx could use threads for file io, and linux file performance became ok as well.
      - mzs 5y ago
        It feels like it's been ten years since I fiddled with this on FreeBSD but EVFILT_AIO worked surprisingly well with aio and kqueue on spinning disk.
      - asveikau 5y ago
        POSIX AIO uses `struct sigevent` to notify completion. On FreeBSD you can set ->sigev_notify to SIGEV_KEVENT, and ->sigev_notify_kqueue to a kqueue descriptor. As for your "performance benefit" question and nobody using it over readiness based approaches: Well yes. No idea how it compares with a traditional readiness based socket program. But it's kind of not the point either. AFAIK the scenario where POSIX AIO makes most sense is files on disk.
- lonjil 5y ago
  There's nothing stopping io_uring from gaining more non-IO operations, and indeed that appears to be the plan.
- asveikau 5y ago
  kqueue is not really a replacement for io_uring. That said, FreeBSD can make a POSIX AIO request signal completion to a kqueue, which is a similar model of async I/O. But as has been pointed out elsewhere, io_uring is also about making other syscalls asynchronous.
binarycrusader 5y ago
Solaris had this over a decade ago now with event ports which are basically a variant of Windows IOCP: https://web.archive.org/web/20110719052845/http://developers.sun.com/solaris/articles/event_completion.html https://web.archive.org/web/20110719052845/http://developers... So at least one UNIX system had them a while ago.
- wahern 5y ago
  People keep saying this but IME that's not how the Solaris Event Ports API works at all. The semantics of Solaris Event Ports is nearly identical to both epoll+family and kqueue. And like with both those others (ignoring io_uring), I/O completion is done using the POSIX AIO interface, which signals completion through the Event Port descriptor. I've written at least two wrapper libraries for I/O readiness, POSIX signal, file event, and user-triggered event polling that encompass epoll, kqueue, and Solaris Event Ports. Supporting all three is relatively trivial from an API perspective because they work so similarly. In fact, notably all three let you poll on the epoll, kqueue, or Event Port descriptor itself. So you can have event queue trees, which is very handy when writing composable libraries.
  - deleted 5y ago
    [deleted]
PaulDavisThe1st 5y ago
Maybe Linux will get scheduler activations in the near future, another OS feature from the 90s that ended up in Solaris and more or less nowhere else. "Let my user space thread scheduler do its work!"
- aseipp 5y ago
  There's already plans for a new futex-based swap_to primitive, for improving userland thread scheduling capabilities. There was some work done on it last year, but it was rejected on LKML. At this rate, it looks like it will not move forward until the new futex2 syscall is in place, since the original API is showing its age. So, it will probably happen Soon™, but you're probably still ~2 years out before you can reliably depend on it, I'd say.
  - PaulDavisThe1st 5y ago
    Scheduler activations don't require swap_to. The kernel wakes up the user space scheduler when it decides to put the process onto a cpu. The user space scheduler decides which user space thread executes in the kernel thread context that it runs in, and does a user space thread switch (not a full context switch) to it. It's a combination of kernel threads and user space (aka "green") threads.
- rektide 5y ago
  It might! I'm not sure if it's an exact fit or not but the User Managed Concurrency Groups work[1] Google is trying to upstream with their Fibers userland-scheduling library sounds like it could be a match, and perhaps it could get the upstreaming itcs seeking. [1] https://www.phoronix.com/scan.php?page=news_item&px=Google-Fibers-Toward-Open https://www.phoronix.com/scan.php?page=news_item&px=Google-F...
  - deleted 5y ago
    [deleted]
- gpderetta 5y ago
  I think some of the *BSDs have (or had) it. Linux almost got it at the turn of the millennium, with the Next Generation Posix Threading project, but then the much simpler and faster NPTL won.
ww520 5y ago
I remember IPX has similar communication model. To read from the network, you post an array of buffer pointers to the IPX driver and can continue to do whatever. When the buffers are filled, the driver calls your completion function.
mpweiher 5y ago
Do you have some references?

joshmarinacci 5y ago

I'm curious how this handles the case where the calling program dies or wants to cancel the request before it's actually happened.

asdfasgasdgasdg 5y ago
I don't know the answer but I would assume if you have submitted an operation to the kernel, you should assume it's in an indeterminate state until you get the result. If the program dies then the call may complete or not. For cancellation there is an API. Example call: https://github.com/axboe/liburing/blob/c4c280f31b0e05a1ea792ce255dc8689aad1c887/test/io-cancel.c#L131 https://github.com/axboe/liburing/blob/c4c280f31b0e05a1ea792...
PaulDavisThe1st 5y ago
Death: same way it handles a program with an open socket and data arriving and unread and the program dies. It's just part of the overall resource set of the process and has to be cleaned up when the process goes away.
ww520 5y ago
The do_exit() function in kernel/exit.c is responsible for general cleanup on a process [1]. Whether a process dies gracefully or abruptly, the kernel calls do_exit() to clean up all the resources owned by the process, like opened files or acquired locks. I would imagine the io_uring related stuff is cleaned up there as well. [1] https://elixir.bootlin.com/linux/v5.13-rc6/source/kernel/exit.c https://elixir.bootlin.com/linux/v5.13-rc6/source/kernel/exi... Edit: I just looked up at the latest version of the source [1]. Yes, it does clean up io_uring related files.

vkka 5y ago

... So it has the potential to make a lot of programs much simpler. More efficient? Yes. Simpler? Not really. A synchronous program would be simpler, everyone who has done enough of these know it.

masklinn 5y ago
I’m guessing they mean that programs which did not want to block on syscalls and had to deploy workarounds can now just… do async syscalls.
- Filligree 5y ago
  Which is every program that wants to be fast, on modern computers. So...
  - masklinn 5y ago
    Nonsense. Let's say your program wants to list a directory, if it has nothing to do during that time then there is no point to using an asynchronous model, that only adds costs and complexity.
    - wtallis 5y ago
      True. But as soon as your program wants to list two directories and knows both names ahead of time, you have an opportunity to fire off both operations for the kernel to work on simultaneously. And even if your program doesn't have any opportunity for doing IO in parallel, being able to chain a sequence of IO operations together and issue them with at most one syscall may still get you improved latency.
      - deleted 5y ago
        [deleted]
      - touisteur 5y ago
        Yes and even clustering often-used-together syscalls... An interesting 2010 thesis https://os.itec.kit.edu/deutsch/2211.php https://os.itec.kit.edu/deutsch/2211.php and https://www2.cs.arizona.edu/~debray/Publications/multi-call.pdf https://www2.cs.arizona.edu/~debray/Publications/multi-call.... for something called 'multi-calls' Interesting times.

tele_ski 5y ago

I think this might be the best explanation I've read of why io_uring should be better than epoll since it effectively collapses the 'tell me when this is ready' with the 'do action' part. That was the really enlightening part for me. I have to say though, the name io_uring seems unfortunate and I think the author touches on this in the article... the name is really an implementation detail but io_uring's true purpose is a generic asynchronous syscall facility that is currently tailored towards i/o. syscall_queue or async_queue or something else...? A descriptive api name and not an implementation detail would probably go a long way in helping the feature be easier to understand. Even window's IOCP seems infinitely better named than 'uring'.

eloff 5y ago
I've seen mixed results so far. In theory it should perform better than epoll, but I'm not sure it's quite there yet. The maintainer of uWebSockets tried it with an earlier version and it was slower. Where it really shines is disk IO because we don't have an epoll equivalent there. I imagine it would also be great at network requests that go to or from disk in a simple way because you can chain the syscalls in theory.
- zxzax 5y ago
  The main benefit to me has been in cases that previously required a thread pool on top of epoll, it should be safe to get rid of the thread pool now and only use io_uring. Socket I/O of course doesn't really need a thread pool in a lot of cases, but disk I/O does.
- eloff 5y ago
  I dug up the thread with the benchmarks: https://github.com/axboe/liburing/issues/189 https://github.com/axboe/liburing/issues/189 io_uring does win on more recent versions, but it's not like it blows epoll out of the water - it's an incremental improvement. Specifically for websockets where you have a lot of idle connections, you don't want a bunch of buffers waiting for reads for long periods of time - this is why Go doesn't perform well for websocket servers as calling Read() on a socket requires tying up both a buffer and a goroutine stack. I haven't looked into how registering buffers works with io_uring, if it's 1-to-1 mapping or if you can pass a pool of N buffers that can be used for reads on M sockets where M > N. The details matter for that specific case. Again where io_uring really shines is file IO because there are no good solutions there currently.

diegocg 5y ago

IIRC the author of io_uring wants to support using it with any system call, not just the current supported subset. Not sure how these efforts are going.

swiley 5y ago
Supporting exec sounds like it could be a security issue. I also have a hard time imagining why on earth you would call exec that way.
- tptacek 5y ago
  Say more about this? You're probably right but I haven't thought about it before and I'd be happy to have you do that thinking for me. :)
  - touisteur 5y ago
    Start a user-provided compression background task at file close? Think tcpdump with io_uring 'take this here buffer you've just read and send it to disk without bothering userland.
- the_duke 5y ago
  > Supporting exec sounds like it could be a security issue. Why would it be any more of an issue than calling a blocking exec? > why on earth you would call exec that way The same reason as why you would want to do anything else with io_uring? In an async runtime you have to delegate blocking calls to a thread pool. Much nicer if the runtime can use the same execution system as for other IO.
  - swiley 5y ago
    >Why would it be any more of an issue than calling a blocking exec? Now you turn everything doing fast I/O into something that's essentially an easier rwx situation WRT memory corruption. >Much nicer In other words: the runtime should handle it because the kernel doesn't need to.

rjzzleep 5y ago

I found this presentation to be quite informative https://www.youtube.com/watch?v=-5T4Cjw46ys https://www.youtube.com/watch?v=-5T4Cjw46ys

ayanamist 5y ago

So linux people find that the model of windows iocp used is better?

hermanradtke 5y ago
As a Linux person: Yes, and I have thought so for a long time. This is why many of us are excited about io_uring.
ot 5y ago
Yes, and now things are going full circle and windows is adopting the ring model too. https://windows-internals.com/i-o-rings-when-one-i-o-operation-is-not-enough/ https://windows-internals.com/i-o-rings-when-one-i-o-operati...
- muststopmyths 5y ago
  huh, interesting. As far as I can tell the main advantage this has over IOCP is that you can get one completion for multiple read requests. Looks like they took a lot of the concepts from Winsock RIO and applied them to file I/O. Which is fascinating because with network traffic you can't predict packet boundaries and thus your I/O rate can be unpredictable. RIO helps you get the notification rate under control, which can help if your packet rate is very high. With files, I would think you can control the rate at which you request data, as well as the memory you allocate for it. The other thing it saves just like RIO is the overhead of locking/unlocking buffers by preregistering them. Is that the main reason for this API then ? I would be very interested to hear from people who have actually run into limits with overlapped file reads and are therefore excited about IoRings
asveikau 5y ago
I think it's long been understood that it's better for disk I/O. I'm not sure the consensus is as clear for sockets.
volta83 5y ago
Yes. Unfortunately Rust went the exact other way.

grok22 5y ago

Isn't locking a problem with io_uring? Won't you block the kernel when it's trying to do the event completion stuff and the completion work tries to take a lock? Or is the completion stuff done entirely in user-space and blocking is not a problem? Maybe I need to read up on this a bit more...

zxzax 5y ago
The submissions and completions are stored in a lock-free ring buffer, hence the name "uring."
- legulere 5y ago
  lock-free ring buffers still have locks for the case when they are full, so it would be interesting to see how the kernel behaves when you never read from the completion ring.
  - zxzax 5y ago
    You will get an error for that upon submit. The application can then buffer the submissions somewhere else and wait for a few completions to finish.
  - jlokier 5y ago
    As long as command submission is only permitted (or considered valid) when the submitter ensures there is sufficient space in the completion ring for the replies, writing to the completion ring can't legitimately block, and the completion writer can assume this. This asymmetry is because the command submitter is able to block and is in control. It even works with elastic ringbuffers that can be expanded as needed. All memory allocation can be on the command submitter side. This trick works in other domains, not just kernels. For example request-reply ringbuffer pairs can be used to communicate with interrupt handlers, in OSes and also in bare metal embedded systems, and the interrupt handler does not need to block when writing to the reply ringbuffer. It's a nice way to queue things without locking issues on the interrupt side. Similarly with signal handlers and attached devices (e.g. NVMe storage, GPUs, network cards), those can can always write to the reply ringbuffer too.

ginsmar 5y ago

Very very interesting.

justsomeuser 5y ago

If my process has async/await and uses epoll, wouldn't this have about the same performance as io_uring? E.g. with io_uring, the event triggers the kernel to read into a buffer. With async/await epoll wakes up my process which does the syscall to read the file. In both cases you still need to read from the device and get the data to the user process?

Misdicorl 5y ago
Imagine your process instead of getting woken up to make a syscall gets woken up with a pointer to a filled data buffer.
- justsomeuser 5y ago
  But some process (kernel or user space) still needs to spend the computers finite resources to read it? If the work happens in the kernel process or user process it still costs the same?
  - Misdicorl 5y ago
    Yes, the syscall work still happens and the raw work has not changed. The overhead has dropped dramatically though since you've eliminated at least 2 context switches per data fill cycle(and probably more).
  - wtallis 5y ago
    The transition from userspace to the kernel and back takes a similar amount of time to actually doing a small read from the fastest SSDs, or issuing a write to most relatively fast SSDs. So avoiding one extra syscall is a meaningful performance improvement even if you can't always eliminate a memcpy operation.
    - Veserv 5y ago
      That sounds unlikely. syscall hardware overhead (entry + exit) on a modern x86 is only on the order of 100 ns which is approximately main memory access latency. I am not familiar with Linux internals or what the fastest SSDs are capable of these days, but I am fairly sure that for your statement to be true Linux would need to be adding 1 to 2 orders of magnitude in software overhead. This occurs in the context switch pathway due to scheduling decisions, but it is fairly unlikely it occurs in the syscall pathway unless they are doing something horribly wrong.

mzs 5y ago

bandwidth exceeded alternative: https://web.archive.org/web/20210617150204/https://despairlabs.com/posts/2021-06-16-io-uring-is-not-an-event-system/ https://web.archive.org/web/20210617150204/https://despairla...

zootboy 5y ago
It seems that their server is rewriting the 451 error to a 403, which caused Archive.org to drop the page from its archives. Unfortunate...

bilalhusain 5y ago

Currently serving Bandwidth Restricted page - 451. Cached version of the write up https://archive.is/VgHkW https://archive.is/VgHkW

mikedilger 5y ago

> io_uring is not an event system at all. io_uring is actually a generic asynchronous syscall facility. I would call it a message passing system, not an asynchronous syscall facility. A syscall, even one that doesn't block indefinitely, transfers control to the kernel. io_uring, once setup, doesn't. Now that we have multiple cores, there is no reason to context switch if the kernel can handle your request on some other core on which it is perhaps already running.

volta83 5y ago
This is how I think about it as well. Which makes me wonder if Mach wasn't right all along.
- mikedilger 5y ago
  I've been intending to try process-to-process message passing using a lock-free data structure very much like io_uring where the processes share pages (one page read-write, the other page readonly, and inverse for the other process) on hardware with L2 tightly-integrated memory (SiFive FU740). The result being zero-copy message passing at L2 cache speeds. [for an embedded realtime custom OS, not for linux]
hinkley 5y ago
It becomes a question of how expensive the message passing is between cores versus the context switch overhead - especially in a world where switching privilege levels has been made expensive by multiple categories of attack against processor speculation. Is there a middle ground where io_uring doesn't require the mitigations but other syscalls do?

raphlinus 5y ago

Something I've been thinking about that maybe the HN hivemind can help with; I know enough about io_uring and GPU each to be dangerous. The roundtrip for command buffer submission in GPU is huge by my estimation, around 100µs. On a 10TFLOPS card, which is nowhere near the top of the line, that's 1 billion operations. I don't know exactly where all the time is going, but suspect it's a bunch of process and kernel transitions between the application, the userland driver, and the kernel driver. My understanding is that games mostly work around this by batching up a lot of work (many dozens of draw calls, for example) in one submission. But it's still a problem if CPU readback is part of the workload. So my question is: can a technique like io_uring be used here, to keep the GPU pipeline full and only take expensive transitions when absolutely needed? I suspect the programming model will be different and in some cases harder, but that's already part of the territory with GPU.

boardwaalk 5y ago
The communication between the host and the GPU already works on a ring buffer on any modern GPU I believe. It’s why graphics APIs are asynchronous until you synchronize f.e. by flipping a frame buffer or reading something back. APIs like Vulkan are very explicit about this and have fences and semaphores. Older APIs will just block if you do something that requires blocking.

bullen 5y ago

To me this sounds like more kernel, I was hoping for less or no kernel instead. On my live machines the kernel is taking 30% CPU copying memory. I'm waiting for userspace networking... file IO can be offset with async-to-async parallelism.