9 ms·
Libuv – Linux: io_uring support
- truth_seeker 3y agoLibraries, VMs, Runtimes backed by libuv : https://github.com/libuv/libuv/blob/v1.x/LINKS.md https://github.com/libuv/libuv/blob/v1.x/LINKS.md
- oaiey 3y agoImpressive. Especially the DNS resolvers. At least the .NET entry is however no longer correct
- loeg 3y ago8x throughput in what benchmark?
- rwmj 3y agoReading small chunks from /dev/zero. Avoiding lots of small syscalls is the best case for io_uring, but also not uncommon in real code.
- gavinray 3y agoThe commenter mentioning reading small chunks from /dev/zero says specifically GREATER than the stated x8 throughput, which makes me think it's x8 in the general sense. I could be mistaken though, it seems odd not to post any sort of numbers/benchmarks.
- TheDong 3y agoI'll eat my files if this gives 8x on normal applications reading normal sized files from real SSDs or HDDs. The commenter mentions "greater than 8x" on a very artificial benchmark of reading /dev/zero. The original PR author than copies that to the PR description as "8x has been observed" (lossily dropping the greater, but not adding any additional datapoints). The reality will be that disk IO dwarfs the overhead of syscalls and threadpools in all real cases. That all that's being saved here: the overhead of having to manage a threadpool, and make a few more syscalls. Compared to reading a few MB off disk, that overhead will not be noticed, and definitely won't be 8x.
- loeg 3y agoI would expect this to benefit IO operations on NVMe media the most, and slow media like HDD the least. The OS stack imposes a substantial part of the overall latency in tiny random operations against NVMe drives.
- alberth 3y agoDoes this mean that NodeJS apps will gain 8x throughout?
- biorach 3y agoIf the app's workload is quite similar to whatever benchmark showed the 8x improvement, then yes. Otherwise, i.e. in the real world, it's very unlikely to be that much. High throughout workloads should see improvement, but each application is different.
- binarymax 3y agoI don’t know why you’re being downvoted, because for most people that don’t work with low level syscalls this is the kind of question they have. From what I understand, some async operations will be faster in node on newer versions of the linux kernel, when node uses a version of libuv that contains this PR.
- truth_seeker 3y agoOnly if they are highly I/O bound. For example, implementing tcp/http/udp proxy servers. Static file servers too can benefit greatly.
- Misdicorl 3y agoNit: Syscall bound, not necessarily I/O bound.
- dontlaugh 3y agoBoth, because it allows async file I/O.
- Misdicorl 3y agoYour application must feel I/O bound. But in reality it must be syscall bound in order for io_uring to make any noticeable improvement. In other words: if your application is already writing/reading at the max speed the hardware its on allows, you will see no improvement from this. That application is truly I/O bound. An application that feels I/O bound will be doing lots and lots of little reads and writes and achieving low hardware utilization. That application is syscall bound rather than I/O bound, and io_uring will help tremendously.
- xiphias2 3y agoMaybe this is an arrogant question, but why adding asyncio to these libraries in general in all OSs is slow? I would think that writing the kernel part would be the hardest, but it's usually the event loop implementations that don't use what the Windows/MacOS/Linux kenels offer.
- rwmj 3y agoWe tried to add io_uring support to libnbd (and indeed still hope to do so). There's a bit of an "impedance mismatch" between existing event-driven code and io_uring, not impossible, but also not completely easy to just convert code. Especially when you want to keep the non-io_uring code working (for old Linux, BSD etc). To give an example, libnbd does a lot of send and recv system calls interleaved, for sending requests and receiving replies, with a complicated state machine to keep track of when we would block. It's still not obvious how to convert this to an io_uring style where you have to add the send/recv requests to the submission queue and then separately pick up the responses from the completion queue. It's a particular problem dealing with possible errors and keeping the API the same (if you just call exit when you hit any error, it's a lot easier). If you're writing all new code then io_uring is a great choice.
- xiphias2 3y agoI was playing with Tokio / Rust on MacOS and hit an about max 500k requests / second limit where the CPU was mostly doing send / recieve calls, not using MacOS asyncio system calls. I know it's a different library, and asyncio is not as flexible as io_uring, but at least batching send / recv calls from different sockets would be good to be able to do.
- woooooo 3y agoLiburing is pretty recent, and it started with a really limited set of syscalls it supports. Everyone is adopting it pretty quickly, actually.
- yxhuvud 3y agoIt is because there in general needs to be pretty big structural changes in the programs to make use of it properly.
- fit2rule 3y ago[dead]
- rektide 3y agoAnd this is after node.js v20 already had some very impressive more compute-centric wins! https://blog.rafaelgss.dev/state-of-nodejs-performance-2023 https://blog.rafaelgss.dev/state-of-nodejs-performance-2023
- gavinray 3y agoVery excited for the performance implications for Node in general. Node is going to be very speedy in v21.
- heyoni 3y agoDoes this potentially mean you could write a sql driver using libuv in python and benefit from async calls performance without the main python scripts using any async libraries or conventions?
- truth_seeker 3y agoasyncio event loop in Python uses `uvloop` which in based on `libuv`
- fanf2 3y agoMy employer, ISC, sponsored this work because we use libuv in BIND9 and we would like everyone to benefit more from the new io_uring hotness.
- 0x000xca0xfe 3y agoAwesome, do you have any data for performance or RAM usage of epoll vs. io_uring in a real-world scenario for BIND9?
- fanf2 3y agoNot yet :-)
- destructionator 3y agoThis test doesn't include epoll in any significant fashion at all, and also doesn't apply to network operations. The libuv thing is specifically about disk files, which linux doesn't really support with epoll (epoll just always considers them ready, regardless of if the requested page is in cache or not, meaning it is not helpful for the async case. by contrast, a socket without data immediately available in the kernel's buffer will not show readiness). Since you can't tell if a disk io function would have to actually hit the disk or use the cache on older linuxes, libuv just always dispatches said operations to a worker thread pool which then calls the normal function. When it returns, it sends a message back to the other thread to indicate its completion. If data is already present in the cache, this adds significant overhead vs just calling `read` directly - instead calling `read`, it will post a request to the worker thread queue (which may require synchronization, im not sure exactly how the impl does it tbh), wake the worker thread which calls `read`, which does not block significantly since the data is indeed already present, then it posts an event back to the main thread (which, sure, it is listening for this event on `epoll`, along with whatever else it is listening to, but that's an insignificant detail at this point) which calls the on-complete handler. All that work instead of just transferring the data from the cache to the result. Significant slowdown. If the data is not already present in the cache, it does exactly those same steps, except this time `read` actually does block for some time, so the main thread can continue with other work in the mean time. (Of course, if the main thread has no other work to do, you've gained nothing from this!) Hence why I said in my other comment that you ought not draw any general conclusion from this. This speedup has nothing to do with epoll vs io_uring (except maybe that linux file i/o is not really epoll compatible) and everything to do with libuv's block device implementation specifically. It is totally inapplicable to network loads entirely. edit: in the first version I said "disk file" but it is technically block devices, of which disk files are just the most common example, but of course /dev/zero is a block device that is not a disk file.... and worth noting that /dev/zero is never going to actually load off a disk meaning it is "in the cache" all the time.
- ithinkso 3y agoIt's the same 'for' in the title that I don't understand in the 'Windows subsystem for Linux'. English is not my native language but it is only recently that I started to notice this usage of for. Has it always been used like this? Github post does it normally: 'Add io_uring support for several asynchronous file operations:'
- rkagerer 3y agoDon't feel bad. Windows Subsystem for Linux was just a horrible name. They couldn't even be bothered to put an apostrophe on the end of Windows.
- deleted 3y ago[deleted]
- gnfargbl 3y ago"X for Y" isn't a new thing ("Windows for Workgroups", 1993) but I think the phrasing is possibly more common in the tech world than outside it.
- kzrdude 3y agoWhat about "German for Engineers"? is that the same "for"? I'm thinking of a ficticious language course for engineers..
- wongarsu 3y agoWhile "Engineers for Ukraine" is probably about building drones and shipping them to the Russian border region.
- ithinkso 3y agoThat's not it, X for Y sounds completely normal as in 'food for kids' or something. But I have read the title as if io_uring added support for libuv.
- 3y ago
- moralestapia 3y agolibuv is such an underrated piece of technology! If you haven't yet, please go check it out, write a program with it and be amazed. So glad to be a contributor.
- oaiey 3y agolibuv brought .NET to Linux. Will never forget that. Without libuv that story would not have been such a success. Thank you!
- goodpoint 3y ago> libuv brought .NET to Linux. Will never forget that. Is that a very good thing or a very bad one?
- 5e92cb50239222b 3y agoIt allowed our shop to drop Windows almost everywhere, so depends on your worldview. The platform itself is nice, although I too don't like the company, and probably never will.
- oaiey 3y agoIt is a good thing. .NET is a wonderful stack nowadays. The company overlord is sometime what he is but generally behaving.
- benatkin 3y agoUnderrated by whom? Is Deno giving up C goodness for Rusty goodness a mistake? I don't think so but it seems to be another time that a big improvement comes with a big drawback. To me libuv seems highly rated and it deserves to be highly rated. I'm not sure how to quantify it exactly to say whether it's fairly rated, underrated, or overrated, but it seems like it's in the ballpark as far as ratings go.
- moralestapia 3y agoI'd add it to the CS curricula, it's that good. Also, a polling loop + how to properly use it should be something as fundamental as pointers, linked lists, ..., all that stuff.
- dan-robertson 3y agoThe 8x speedup seems to come from a microbenchmark that requires no real async work from the kernel (so I think is mostly going to be stressing out context switches and the threadpool data structures in the non-uring case) but I’m still excited about the improvements to async io from io_uring. Question I’ve not figured out yet: how can one trace io_uring operations? The api seems kinda incompatible with ptrace (which is what strace uses) but maybe there is an appropriate place to attach an ebpf? Or maybe users of io_uring will have to add their own tracing?
- truth_seeker 3y agoObservability on how kernel handles io_uring request - https://github.com/axboe/liburing/issues/467 https://github.com/axboe/liburing/issues/467
- tptacek 3y agoYes: the title of this post should be "libuv: introduce io_uring support", not what it is now.
- znpy 3y ago> The 8x speedup seems to come from a microbenchmark that requires no real async work from the kernel which is nice because you can measure the actual library overhead without the noise from real kernel work.
- theLiminator 3y agoYeah, might have been better to reword it as a "8x reduction in overhead" or something to make it more clear that it's highly unlikely you'll see this in real workloads.
- 29athrowaway 3y agoAdd "On Linux" to the title.
- speed_spread 3y agoio_uring, as mentioned in the title, implies Linux.
- 29athrowaway 3y agoThe title was changed by the time you replied.
- MichaelMoser123 3y agovery impressive! Just one questions: what about older versions of linux that don't have io_uring, does it fall back gracefully to older system calls or are these older versions of linux no longer supported?
- truth_seeker 3y agoYep. Auto Fallback to older Linux api is in place already.
- destructionator 3y agolibuv's implementation of "async" disk i/o brings massive overhead (somewhat necessarily on linux, totally unnecessarily on other systems), so just about anything (including just switching to straight blocking read/write/seek/etc syscalls) would result in a significant speed increase. This isn't to say that io_uring is bad, just don't draw too much a conclusion from any benchmark of their old impl beyond the context of their old impl specifically.
- Ahmad498 3y agoThis is ready for review now. Linux v5.13 is the minimum required, otherwise libuv falls back to the thread pool.
- errantmind 3y ago[flagged]
- samsquire 3y agoThis is really good. Thank you! I've been studying how to create an asynchronous runtime that works across threads. My goal: neither CPU and IO bound work slow down event loops. How do you write code that elegantly defines a state machine across threads/parallelism/async IO? How do you efficiently define choreographies between microservices, threads, servers and flows? I've only written two Rust programs but in Rust you presumably you can use Rayon (CPU scheduling) and Tokio (IO scheduling) I wrote about using the LMAX Disruptor ringbuffer pattern between threads. https://github.com/samsquire/ideas4#51-rewrite-synchronous-code-into-lmax-disruptor-thread-pools---event-loops-that-dont-block-on-cpu-usage https://github.com/samsquire/ideas4#51-rewrite-synchronous-c... I am designing a state machine formulation syntax that is thread safe and parallelises effectively. It looks like EBNF syntax or a bash pipeline. Parallel steps go in curly brackets. There is an implied interthread ringbuffer between pipes. It is inspired by prolog, whereby there can be multiple conditions or "facts" before a stateline "fires" and transitions. Transitions always go from left to right but within a stateline (what is between a pipe symbol) can fire in any order. A bit like a countdown latch. states = state1 | {state1a state1b state1c} {state2a state2b state2c} | state3 You can can think of each fact as an "await" but all at the same time. initial_state.await = { state1a.await state1b.await state1c.await }.await { state2a.await state2b.await state2c.await } | state3.await In io_uring and LMAX Disruptor, you split all IO into two halves: submit and handle. Here is a liburing state machine that can send and receive in parallel. accept | { submit_recv! | recv | submit_send } { submit_send! | send | submit_recv } I want there to be ring buffers between groups of states. So we have full duplex sending and receiving. Here is a state machine for async/await between threads: next_free_thread = 2 task(A) thread(1) assignment(A, 1) = running_on(A, 1) | paused(A, 1) running_on(A, 1) thread(1) assignment(A, 1) thread_free(next_free_thread) = fork(A, B) | send_task_to_thread(B, next_free_thread) | running_on(B, 2) paused(B, 1) running_on(A, 1) | { yield(B, returnvalue) | paused(B, 2) } { await(A, B, returnvalue) | paused(A, 1) } | send_returnvalue(B, A, returnvalue)
- gigatexal 3y agoreally cool to read that thread and see neovim devs looking at it as well. we need a sort of open source hall of fame and axboe should be there for sure
- quietbritishjim 3y ago> Add io_uring support for several asynchronous file operations: read, write; fsync, fdatasync; stat, fstat, lstat Does this mean libuv already supported io_uring for non-file operations? Or it still doesn't? Async file operations are useful in some applications, but not the main things people normally think of when they hear async IO.