6 ms·

I’m no kernel hacker, but it seems like io_uring is almost being undersold despite the hype. “Async IO” seems like an understatement. It’s (to my meager knowled

by klabb3 4y ago

I’m no kernel hacker, but it seems like io_uring is almost being undersold despite the hype. “Async IO” seems like an understatement. It’s (to my meager knowledge) a general purpose syscall batching interface. There could even be more interesting use-cases (like IPC?) in the future. The genius (and in hindsight almost obvious) idea behind io_uring seems too good to ignore, which makes me hopeful about cross-platform adoption in the long run. That would enable huge simplifications in language runtimes and low-level frameworks, lowering the barrier of entry to high performance IO across the board.

philosopher1234 4y ago
You cant leave me hanging!! Whats the genius idea?
- mattclarkdotnet 4y ago
  “General purpose syscall batching”, which drastically cuts down on context switches
- shorodei 4y ago
  You could use the shared ring buffers scheme to replace essentially all syscalls and similar things in any context. Think gpu drivers, memory controllers, etc. It could be a universal interface for all communication that has to go through some kind of expensive security barrier.
  - vluft 4y ago
    With cross-core interrupts and user-mode interrupt handlers (as in some new intel cpus), you could even do something without polling (interrupt for submission) where the core user-mode code is running on _never_ context switches (obviously except for scheduling) and you just have a dedicated kernel core or cores off doing kernel things.
    - benlwalker 4y ago
      You can also use umwait on the next completion entry in the ring
      - vluft 4y ago
        yup, though that means you're wasting that core's compute; something with green threads where language runtime does a cross-core interrupt to submit syscall then continues execing other green threads until it gets a user interrupt for syscall completion would be pretty neat. (ETA: and indeed looks like they're considering support for that in io_uring! https://lwn.net/Articles/869140/ https://lwn.net/Articles/869140/)
- benlwalker 4y ago
  Imagine you have a piece of software that runs in an event loop (as many things do). On each loop, queue up all system calls you'd like to perform. At the end of the loop, do one syscall to execute the batch. At the start of the loop, check if anything has completed and continue the operation. If you're processing a set of sockets and on any given loop N are ready, then with epoll you do N+1 syscalls. With io_uring you do 1. It's independent of N.
  - klabb3 4y ago
    And the potential impact is huge! Not only are individual syscalls expensive on their own, and increasingly so (afaik) with spectre and security issues. You also need a thread to execute the call, which is several KB of memory, compound context switches, and (often overlooked) creating and destroying threads also come with syscall overhead. Now, we’ve had epoll etc so it’s not novel in that respect. However, what’s truly novel is that it’s universal across syscalls, which makes it almost mechanical to port to a new platform. A lot of intricate questions of high-level API design simply go away and become more simple data layout questions. (I’m sure there are little devils hiding in the details, but still)
rektide 4y ago
Weirdly the low barrier batched sys calls & ebpf user-programmable-kernel sort of feel like a comeback-ish story for un-monolothic kernels/OSes. After a decade+ of it feeling like Linus/Tannenbaum debate was a de jure settled debate. The debate being anchored to either/or certainly didn't help, confined the dimensionality of what we might consider, eliminated vaster exploration of the possible. But the idea that it's all in the kernel seems improbably from the modern outlook (with some of the GPL IP copy-left awesomeness as notable sacrifice, which has yet to really manifest, but IMO hugely looms). Again and again I keep thinking we mistake popularity & rapid adoption for success. There's so many brilliant wonderful possibilities. People & attitudes coallesce to the negative enormously fastyl, with much more emotion than belief spreads. Finding ways to give long time for tech to slowly grow & quietly build & thrive is a huge huge challenge. Uring is definitely a nice example of a small conceptual extension of what is that can snowball & grow & metastatize, until it far outshines everything we did before. Now if only Deno, Node & others would start getting more of this shipped! That said, slow & right is what I'm defending here, soooo, OK.
- softirq 4y ago
  This really has nothing to do with the mono vs. micro kernel argument. Also micro kernels were very popular in the research community at the time that Linux was in its infancy, mach and minix are both examples of this. Linux did win on merit and the future work of micro kernels was forked and past micro kernels are only vestigially micro. Edit: Not sure why this is being downvoted, micro kernels are the antithesis of what eBPF is trying to accomplish. I say this as a university researcher that has built and worked on many kernels with different tradeoffs. If you disagree please explain in a comment. Micro kernels increase security and decrease complexity by running a very small resource manager in the processor's highest privilege level. eBPF runs code at the highest privilege level by actually extending the kernel, it actually supplants doing kernel work in userspace. It greatly increases kernel complexity with an new subsystem and decreases security in order to allow extensibility in the face of a hard to change interface (syscalls in a mature operating systems are difficult to change).
  - rektide 4y ago
pcl 4y ago
It’s worth looking at the work that Martin Thompson has done with similar techniques in the JVM space with Aeron and Disruptor: https://aeron.io/ https://aeron.io/ https://lmax-exchange.github.io/disruptor/ https://lmax-exchange.github.io/disruptor/
bfrog 4y ago
It absolutely a batch interface for syscalls, with the ability to sort of create little I/O specific programs.
api 4y ago
If it batches syscalls could libc use it in place of the normal syscall method? You’d have to kind of deliberately nerf it by waiting on calls in spite of its async nature but this might still bring a boost due to reduced kernel user mode switching and other overhead.
skavi 4y ago
interesting whitequark thread on that idea: https://twitter.com/whitequark/status/1521146654535172100 https://twitter.com/whitequark/status/1521146654535172100
epage 4y ago
Whats old becomes new again. I worked on some Windows drivers that batched calls into the kernel because of the overhead "back in the day". At the time, we were re-working the drivers to not doing batching anymore because the overhead of syscalls had dropped (as part of a larger clean up of our driver design, not a reason in of itself).
- kps 4y ago
  > Whats old becomes new again. The polled mode of io_uring resembles I/O on CDC mainframes, except that the ‘kernel’ ran on dedicated coprocessors (which incidentally ran multiple tasks in a way similar to what is now called hyperthreading).
- espoal 4y ago
  io_uring it's not about batching. io_uring is about a new concurrency model based on ring buffers
- fulafel 4y ago
  Before that in the 60s io command batching was (like most things) done by IBM: https://en.wikipedia.org/wiki/Channel_I/O#Channel_program https://en.wikipedia.org/wiki/Channel_I/O#Channel_program These days Linux supports VFIO pass-through for native IBM/360 channel programs if you happen to be running Linux on a IBM mainframe: https://docs.kernel.org/6.1/s390/vfio-ccw.html https://docs.kernel.org/6.1/s390/vfio-ccw.html
ddevault 4y ago
I am a kernel hacker, and I have worked with io_uring, and I can safely judge that it is very good -- but the main issue is that it represents a totally different approach to I/O (and syscalls generally, which are just I/O in other words), which is going to take the ecosystem a while to reform around. Note that the sample code in the OP's article is much more complex than the traditional approach. It's also very Linux-specific, so any software which takes advantage of it will be less portable or will have to write multiple I/O backends. It's also nontrivial to understand and use effectively, so adding good io_uring support to a project is an effort.
- another2another 4y ago
  My hope is that Linux ports of kqueue or libdispatch can use io_uring where possible and we would automatically get the benefit, but I'm not familiar enough with the guts of these projects to know how timely or even feasible that is.
flohofwoe 4y ago
It seems to be the same evolution that we've seen in 3D rendering APIs (or more generally: GPU APIs) over the last two decades, and roughly for the same reasons (but it goes beyond just reducing syscall overhead, it's also establishing a pipeline for sending batches of work prepared by the CPU for 'decoupled' execution on the GPU, e.g. trading latency for better throughput).
- fwsgonzo 4y ago
  Yep, command queues are basically how drivers have been for decades now anyway. It just happens to be more widely exposed in user-facing APIs now. If you can call them user-facing APIs. They are hard to use and get right, and nobody writes directly against them. Instead it's best done through a higher level API (eg. an asynchronous system call API) or even higher level through not even knowing about it (eg. modern game engines). Modern caches are heavily pipelined in order to saturate links and well, as you probably know, games are 1-2 frames behind but do tens of thousands of GPU operations per frame. I guess we can even add modern CPUs to the list? Eg. certain operations cause a pipeline flush just like some rendering operations can cause the same thing, under certain circumstances. I just found this article about the graphics pipeline from 2011: https://fgiesen.wordpress.com/2011/07/02/a-trip-through-the-graphics-pipeline-2011-part-2/ https://fgiesen.wordpress.com/2011/07/02/a-trip-through-the-...

by klabb3 4y ago

philosopher1234 4y ago

You cant leave me hanging!! Whats the genius idea?

mattclarkdotnet 4y ago
“General purpose syscall batching”, which drastically cuts down on context switches
shorodei 4y ago
You could use the shared ring buffers scheme to replace essentially all syscalls and similar things in any context. Think gpu drivers, memory controllers, etc. It could be a universal interface for all communication that has to go through some kind of expensive security barrier.
- vluft 4y ago
  With cross-core interrupts and user-mode interrupt handlers (as in some new intel cpus), you could even do something without polling (interrupt for submission) where the core user-mode code is running on _never_ context switches (obviously except for scheduling) and you just have a dedicated kernel core or cores off doing kernel things.
  - benlwalker 4y ago
    You can also use umwait on the next completion entry in the ring
    - vluft 4y ago
      yup, though that means you're wasting that core's compute; something with green threads where language runtime does a cross-core interrupt to submit syscall then continues execing other green threads until it gets a user interrupt for syscall completion would be pretty neat. (ETA: and indeed looks like they're considering support for that in io_uring! https://lwn.net/Articles/869140/ https://lwn.net/Articles/869140/)
benlwalker 4y ago
Imagine you have a piece of software that runs in an event loop (as many things do). On each loop, queue up all system calls you'd like to perform. At the end of the loop, do one syscall to execute the batch. At the start of the loop, check if anything has completed and continue the operation. If you're processing a set of sockets and on any given loop N are ready, then with epoll you do N+1 syscalls. With io_uring you do 1. It's independent of N.
- klabb3 4y ago
  And the potential impact is huge! Not only are individual syscalls expensive on their own, and increasingly so (afaik) with spectre and security issues. You also need a thread to execute the call, which is several KB of memory, compound context switches, and (often overlooked) creating and destroying threads also come with syscall overhead. Now, we’ve had epoll etc so it’s not novel in that respect. However, what’s truly novel is that it’s universal across syscalls, which makes it almost mechanical to port to a new platform. A lot of intricate questions of high-level API design simply go away and become more simple data layout questions. (I’m sure there are little devils hiding in the details, but still)

rektide 4y ago

Weirdly the low barrier batched sys calls & ebpf user-programmable-kernel sort of feel like a comeback-ish story for un-monolothic kernels/OSes. After a decade+ of it feeling like Linus/Tannenbaum debate was a de jure settled debate. The debate being anchored to either/or certainly didn't help, confined the dimensionality of what we might consider, eliminated vaster exploration of the possible. But the idea that it's all in the kernel seems improbably from the modern outlook (with some of the GPL IP copy-left awesomeness as notable sacrifice, which has yet to really manifest, but IMO hugely looms). Again and again I keep thinking we mistake popularity & rapid adoption for success. There's so many brilliant wonderful possibilities. People & attitudes coallesce to the negative enormously fastyl, with much more emotion than belief spreads. Finding ways to give long time for tech to slowly grow & quietly build & thrive is a huge huge challenge. Uring is definitely a nice example of a small conceptual extension of what is that can snowball & grow & metastatize, until it far outshines everything we did before. Now if only Deno, Node & others would start getting more of this shipped! That said, slow & right is what I'm defending here, soooo, OK.

softirq 4y ago
This really has nothing to do with the mono vs. micro kernel argument. Also micro kernels were very popular in the research community at the time that Linux was in its infancy, mach and minix are both examples of this. Linux did win on merit and the future work of micro kernels was forked and past micro kernels are only vestigially micro. Edit: Not sure why this is being downvoted, micro kernels are the antithesis of what eBPF is trying to accomplish. I say this as a university researcher that has built and worked on many kernels with different tradeoffs. If you disagree please explain in a comment. Micro kernels increase security and decrease complexity by running a very small resource manager in the processor's highest privilege level. eBPF runs code at the highest privilege level by actually extending the kernel, it actually supplants doing kernel work in userspace. It greatly increases kernel complexity with an new subsystem and decreases security in order to allow extensibility in the face of a hard to change interface (syscalls in a mature operating systems are difficult to change).
- rektide 4y ago

pcl 4y ago

It’s worth looking at the work that Martin Thompson has done with similar techniques in the JVM space with Aeron and Disruptor: https://aeron.io/ https://aeron.io/ https://lmax-exchange.github.io/disruptor/ https://lmax-exchange.github.io/disruptor/

bfrog 4y ago

It absolutely a batch interface for syscalls, with the ability to sort of create little I/O specific programs.

api 4y ago

If it batches syscalls could libc use it in place of the normal syscall method? You’d have to kind of deliberately nerf it by waiting on calls in spite of its async nature but this might still bring a boost due to reduced kernel user mode switching and other overhead.

skavi 4y ago

interesting whitequark thread on that idea: https://twitter.com/whitequark/status/1521146654535172100 https://twitter.com/whitequark/status/1521146654535172100

epage 4y ago

Whats old becomes new again. I worked on some Windows drivers that batched calls into the kernel because of the overhead "back in the day". At the time, we were re-working the drivers to not doing batching anymore because the overhead of syscalls had dropped (as part of a larger clean up of our driver design, not a reason in of itself).

kps 4y ago
> Whats old becomes new again. The polled mode of io_uring resembles I/O on CDC mainframes, except that the ‘kernel’ ran on dedicated coprocessors (which incidentally ran multiple tasks in a way similar to what is now called hyperthreading).
espoal 4y ago
io_uring it's not about batching. io_uring is about a new concurrency model based on ring buffers
fulafel 4y ago
Before that in the 60s io command batching was (like most things) done by IBM: https://en.wikipedia.org/wiki/Channel_I/O#Channel_program https://en.wikipedia.org/wiki/Channel_I/O#Channel_program These days Linux supports VFIO pass-through for native IBM/360 channel programs if you happen to be running Linux on a IBM mainframe: https://docs.kernel.org/6.1/s390/vfio-ccw.html https://docs.kernel.org/6.1/s390/vfio-ccw.html

ddevault 4y ago

I am a kernel hacker, and I have worked with io_uring, and I can safely judge that it is very good -- but the main issue is that it represents a totally different approach to I/O (and syscalls generally, which are just I/O in other words), which is going to take the ecosystem a while to reform around. Note that the sample code in the OP's article is much more complex than the traditional approach. It's also very Linux-specific, so any software which takes advantage of it will be less portable or will have to write multiple I/O backends. It's also nontrivial to understand and use effectively, so adding good io_uring support to a project is an effort.

another2another 4y ago
My hope is that Linux ports of kqueue or libdispatch can use io_uring where possible and we would automatically get the benefit, but I'm not familiar enough with the guts of these projects to know how timely or even feasible that is.

flohofwoe 4y ago

It seems to be the same evolution that we've seen in 3D rendering APIs (or more generally: GPU APIs) over the last two decades, and roughly for the same reasons (but it goes beyond just reducing syscall overhead, it's also establishing a pipeline for sending batches of work prepared by the CPU for 'decoupled' execution on the GPU, e.g. trading latency for better throughput).

fwsgonzo 4y ago
Yep, command queues are basically how drivers have been for decades now anyway. It just happens to be more widely exposed in user-facing APIs now. If you can call them user-facing APIs. They are hard to use and get right, and nobody writes directly against them. Instead it's best done through a higher level API (eg. an asynchronous system call API) or even higher level through not even knowing about it (eg. modern game engines). Modern caches are heavily pipelined in order to saturate links and well, as you probably know, games are 1-2 frames behind but do tens of thousands of GPU operations per frame. I guess we can even add modern CPUs to the list? Eg. certain operations cause a pipeline flush just like some rendering operations can cause the same thing, under certain circumstances. I just found this article about the graphics pipeline from 2011: https://fgiesen.wordpress.com/2011/07/02/a-trip-through-the-graphics-pipeline-2011-part-2/ https://fgiesen.wordpress.com/2011/07/02/a-trip-through-the-...