8 ms·
Using io_uring for network I/O
- victor106 4y agodoes anyone here know how to use this with Java? Does Java by default use io_uring?
- mfiguiere 4y agoio_uring will indeed be a perfect fit for Java: no need to pay a supplemental JNI access cost for each IO as all you need is a memory barrier to read or write the shared queues which can be properly implemented in pure Java. We're not there yet but here is where you can look: - For network I/O, Netty has an incubating transport that is promising [1]. - For disk I/O, JDK's Loom project [2] has mentioned its plan to rely on io_uring on Linux [3], but there's no ETA AFAIK. [1] https://github.com/netty/netty-incubator-transport-io_uring https://github.com/netty/netty-incubator-transport-io_uring [2] https://openjdk.org/projects/loom/ https://openjdk.org/projects/loom/ [3] https://cr.openjdk.org/~rpressler/loom/loom/sol1_part1.html https://cr.openjdk.org/~rpressler/loom/loom/sol1_part1.html
- rotifer 4y agoI don't believe that Java currently uses it by default, although I've seen discussion about possibly using it in the future. However, you might find these links useful: - I built an ultra high performance HTTP server in Java, powered by io_uring - https://old.reddit.com/r/java/comments/12f2h79/i_built_an_ultra_high_performance_http_server_in/ - server is here - https://github.com/bbeaupain/hella-http - based on these bindings - https://github.com/bbeaupain/nio_uring I wouldn't be surprised if there are others.
- Thaxll 4y agoNo languages use io uring by default, it's pretty much work in progress at best.
- HL33tibCe7 4y agoHN’s title “fixer” takes another victim. “Why you should use io_uring for network I/O” is not semantically the same as “Use io_uring for network I/O”. You can’t just swap one out for the other.
- fatneckbeard 4y ago[flagged]
- pcl 4y agoFWIW, the submitter of a post can edit the title even after the “fixer” runs.
- quickthrower2 4y agoThat relies on them spotting it, caring about it, knowing that they can edit it and not worried about undoing an automod operation.
- deleted 4y ago[deleted]
- pcl 4y agoYeah — I’m not suggesting it’s perfect. Just highlighting what’s possible, since I didn’t discover that capability myself until like 3 weeks ago..!
- mfiguiere 4y agoI've actually edited the title back to the original title after the "fixer" changed it. But it's been later "fixed" again. Looks like there's sometimes a second pass of fixing after which edits are no longer possible.
- heyoni 4y agoI guess the difference is that there’s an implied alternative?
- 4y ago
- DominoTree 4y agoio_uring is staggeringly fast, and still seeing massive improvements regularly. Their work in related subsystems is also yielding some fantastic gains: https://www.phoronix.com/news/Linux-Pipes-IOCB_NOWAIT https://www.phoronix.com/news/Linux-Pipes-IOCB_NOWAIT
- eatonphil 4y agoJens Axboe's Twitter is another great place to follow for improvements. https://twitter.com/axboe https://twitter.com/axboe
- candiddevmike 4y agoStill waiting on io_uring in the Go stdlib: https://github.com/golang/go/issues/31908 https://github.com/golang/go/issues/31908 "Unplanned"
- slashdev 4y agoLikewise for Mio (which underlies Tokio) for Rust. The odds of that happening look better, but still seems far away.
- jeroenhd 4y agoWhile Mio will probably not implement uring in its current design, there's https://github.com/tokio-rs/tokio-uring https://github.com/tokio-rs/tokio-uring if you want to use io_uring in Rust. It's still in development, but the Tokio team seems intent on getting good io_uring support at least! As the README states, the Rust implementation requires a kernel newer than the one that shipped with Ubuntu 20.04 so I think it'll be a while before we'll see significant development among major libraries.
- slashdev 4y agoVery interesting! Thanks for sharing.
- tomohawk 4y agoWe've been able to do a lot with this: https://github.com/dshulyak/uring https://github.com/dshulyak/uring
- jerf 4y agoNew queuing mechanisms are generally much bigger shocks than may meet the eye, and it's unusual to see existing systems and frameworks be able to take full advantage of them immediately. In this case, the Go runtime is playing the part of a "framework"; it has ideas about how system calls work and those ideas come from a pre-io_uring world. IIRC, the same thing happened back when epoll was new. You couldn't just swap out your select-based framework to epoll with no changes and just magically get the new performance; it required a lot of work and years for things to finally get polished up. It always seems so easy at the high level, but these abstractions are at the very foundation of the system and the slightest change ripples everywhere. The link you show provides a great demonstration of that; the first paragraph is an optimistic "it seems feasible", then there's literally pages of "umm, well, if you think about this it's a bit more complicated than it seems" and cross-platform issues and upgrades to the kernel mechanism that solved other people's problems maybe solving these problems too and details details details. That's not to say it won't ever happen. Like I said, epoll and other similar things all went through the same basic process, so it can be done, has been done, and will be done again (if not in Go, then in other places). It's just that it's all so much harder than it looks from 30,000 feet in the air.
- klabb3 4y agoI’m no kernel hacker, but it seems like io_uring is almost being undersold despite the hype. “Async IO” seems like an understatement. It’s (to my meager knowledge) a general purpose syscall batching interface. There could even be more interesting use-cases (like IPC?) in the future. The genius (and in hindsight almost obvious) idea behind io_uring seems too good to ignore, which makes me hopeful about cross-platform adoption in the long run. That would enable huge simplifications in language runtimes and low-level frameworks, lowering the barrier of entry to high performance IO across the board.
- philosopher1234 4y agoYou cant leave me hanging!! Whats the genius idea?
- mattclarkdotnet 4y ago“General purpose syscall batching”, which drastically cuts down on context switches
- shorodei 4y agoYou could use the shared ring buffers scheme to replace essentially all syscalls and similar things in any context. Think gpu drivers, memory controllers, etc. It could be a universal interface for all communication that has to go through some kind of expensive security barrier.
- vluft 4y agoWith cross-core interrupts and user-mode interrupt handlers (as in some new intel cpus), you could even do something without polling (interrupt for submission) where the core user-mode code is running on _never_ context switches (obviously except for scheduling) and you just have a dedicated kernel core or cores off doing kernel things.
- benlwalker 4y agoYou can also use umwait on the next completion entry in the ring
- medler 4y agoA few months back I tried using io_uring for some performance-critical network I/O and found it was slower than epoll. A bit sad because epoll has a notoriously janky API and the io_uring API is much nicer. (This part is also sad for me as a Unix/linux fanboy because io_uring’s API is very similar to what Windows was doing 20 years ago).
- dontlaugh 4y agoIOCP was indeed ahead of its time and should have been copied by Linux a long time ago.
- benlwalker 4y agoIOCP certainly was ahead of its time, but it only does the completion batching, not the submission batching. io_uring is significantly better than anything available on Windows right now.
- dontlaugh 4y agoSure, io_uring is better. IOCP still captured the biggest benefit, completion triggering.
- benlwalker 4y agoWithout the submission batching you lose your system call reduction and that is far and away the biggest benefit.
- dontlaugh 4y agoThe way I see it, a bigger benefit is avoiding readiness-triggered IO. That can add a variable number of syscalls, wakes up threads, etc. Of course, batching is also great.
- mappu 4y ago> io_uring is significantly better than anything available on Windows right now. Windows 11 copied io_uring almost 1:1 - https://windows-internals.com/ioring-vs-io_uring-a-comparison-of-windows-and-linux-implementations/ https://windows-internals.com/ioring-vs-io_uring-a-compariso...
- 0xbadcafebee 4y agoOr, don't, and have your apps still work just fine (and be portable)
- eru 4y agoShouldn't you be able to hide that behind a library?
- 0xbadcafebee 4y agoYou could say that about any software at all. The point is, you don't need to use io_uring. Almost nobody does.
- eru 4y agoNo, you can't necessarily say that about any software at all. Eg the style differences between blocking and non-blocking IO are big enough, that it's hard to hide them behind a library.
- rektide 4y agoIdeally support gets built into languages & users just get a free performance boost without even thinking about it. Wouldn't that be great? Bun has io-uring support. Deno issue, https://github.com/denoland/deno/issues/16232 https://github.com/denoland/deno/issues/16232 . Node.js issue with prototype, https://github.com/libuv/libuv/issues/1947 https://github.com/libuv/libuv/issues/1947. Getting good seems interesting & compelling to me. I'm not quite sure where your dislike of this better way of doing things stems from? Even if it does mean changing how we program a bit, to actually go about adopting a better way.
- eru 4y ago> Ideally support gets built into languages [...] Depends on how narrow you want to make your definitions of 'language'. If a library can handle things ergonomically and fast, no need to bake it into the language itself. Eg Perl has regular expressions built in, but Python handles them via a library (arguably it's via the standard library). More drastically: C has a few loops built in, and Haskell handles arbitrary loops via a library and you can build your own new constructs, too. See https://hackage.haskell.org/package/monad-loops-0.4.3/docs/Control-Monad-Loops.html https://hackage.haskell.org/package/monad-loops-0.4.3/docs/C... (Incidentally, that's why Haskell is one of the best imperative languages.)
- languagehacker 4y agoIn practice it's fast but you better hope you've validated it on a stable image with a consistent BOM. If that doesn't match your environment, beware!
- mort96 4y agoBOM?
- bricss 4y agolibuv -> linux: introduce io_uring support https://github.com/libuv/libuv/pull/3952 https://github.com/libuv/libuv/pull/3952
- ptrwis 4y agoFinally something happens
- rektide 4y agoHat's off for posting this 2 hours after it dropped! I've been tracking the nest of issues with anticipation! This wasn't linked to https://github.com/libuv/libuv/pull/1947 https://github.com/libuv/libuv/pull/1947 when it posted, so I didn't see it. Very glad you linked it, thanks! Oh wowwwww.... 608 lines. It's tiny! The last go at adding io_uring to libuv was 24k lines. https://github.com/libuv/libuv/pull/2322 https://github.com/libuv/libuv/pull/2322
- synack 4y agoMy initial impression of io_uring a few years ago was that it involves a lot of pointer manipulation in userspace, leaving a lot of room for alignment and overflow bugs. ...or maybe I'm just being paranoid.
- softirq 4y agoio_uring (and eBPF for that matter) are still super concerning attack vectors and we've already seen eBPF being used in the wild by bad actors. kernel_lockdown explicitly disables eBPF and other methods of kernel injection and is turned on by secure boot. In io_uring's case sharing a userspace mapping with kernel space is simply always going to be dangerous. End of story. The jury is also out as to what the performance benefits of io_uring are and whether the benefits outweigh the risks versus sticking to what's available. I've seen a lot of good results, but I've also seen bad results, and I haven't seen a lot of real world data at scale yet.
- hedora 4y agoThe io_uring API is fairly narrow. It's basically the same interface as the kernel uses to talk to network cards and modern disks. Those are untrusted just like userspace is. You can look at the packet format here (section 4.1): https://kernel.dk/io_uring.pdf https://kernel.dk/io_uring.pdf It's not hard to parse, and nowhere near the level of exposure eBPF creates. If you don't trust the hardware MMU to allow the kernel to safely read buffers from userspace, then there's really no way to perform I/O in the first place. (write() already does this, for example).
- softirq 4y agoThe kernel talks to NICs in kernel mode and can actually segregate a device's view of memory via an iommu. While there's some overlap in potential vulnerabilities, bad hardware/firmware is a different vector than userland having a shared mapping active to use in exploits that read arbitrary kernel data. io_uring is also very complex. It's now it's own subsystem, has it's own worker pool, and even the dance of the rings themselves moving pointers around and using data structure that must be manipulated from both sides is not simple and thus probably not that secure.
- firstlink 4y ago> In io_uring's case sharing a userspace mapping with kernel space is simply always going to be dangerous. Out of curiosity, have you ever used a syscall which writes or reads a userspace buffer?
- kbd 4y agoI learned about io_uring from King Protty in his super informative presentation: https://youtu.be/Ul8OO4vQMTw https://youtu.be/Ul8OO4vQMTw
- rossmohax 4y agoThere is still no good support for io_uring in Rust. AFAIK none of existing solutions fully support multishot ops, zerocopy ops with delayed buffer release, sending messages to another ring with IORING_OP_MSG_RING or convenient ops linking
- lordnacho 4y agoThe Tokio discord seems to have a channel for it. That might be a place to ask. I was just going to wait to see how they do with it.
- conradludgate 4y agoBecause memory is logically shared between the kernel and the application, it makes managing the ownership of memory in safe rust tricky! So unfortunately it's not as plug and play as one would hope
- iamwil 4y agoOne interesting thing is that io_uring can operate in different modes. One of the modes enables kernel-side polling, so that when you put data into the buffer, the kernel will pull the data out itself to do IO. That means from the application side, you can perform IO without any system calls. Our general take was also that it has a lot of potential, but is relatively low level that most mainstream programmers aren't going to pay attention to it. Hence, it'll be a while before it permeates through various ecosystems. For those of you that like to listen on the way to work, we cover io_uring on our podcast, The Technium. https://www.youtube.com/watch?v=Ebpnd7rPpdI https://www.youtube.com/watch?v=Ebpnd7rPpdI https://open.spotify.com/episode/3MG2FmpE3NP7AK7zqQFArE?si=sKNKcrIZRayMeQGTbwkMlw https://open.spotify.com/episode/3MG2FmpE3NP7AK7zqQFArE?si=s...
- gigatexal 4y agoio_uring is really amazing work. I've followed it's development on Twitter. I am in awe of Jens Axboe, I find him very easy to talk to and his work really amazing. If you don't know he's also the man behind the fio disk benchmark utility. Any linux/unix wizards here have any inkling of if or when or how likely it would be for common parts of tools or subsystems etc of the unix/linux stack that we use to integrate io_uring and move away from older async frameworks?
- espoal 4y agoI will add this to my awesome collection: https://github.com/espoal/awesome-iouring https://github.com/espoal/awesome-iouring
- ryanschneider 4y agoThe last strace output (where the author mentions logging) made me think: are there any logging frameworks that’s use io_uring? Seems like a potentially good way to lower logging overhead without adding a lot of buffering to the logging framework.
- donaldihunter 4y agoAuthor here. Good point. It should be easy enough for me to change my test code to use io_uring for logging as well. It will be interesting to see the results.
- xxpor 4y agoThe real killer will be when it's added as an option to iperf3.
- sfink 4y agoI hope there is (or will be) a way to use io_uring for the sorts of purposes that the syscall boundary is currently used for. The example I have in mind is https://rr-project.org/ https://rr-project.org/ which uses the syscall boundary to isolate nondeterminism in order to implement record/replay debugging. I'm not sure how it can accomplish the same thing with io_uring; it would need to be informed of all updates to the shared pages or something?