12 ms·

lsr: ls with io_uring

351 points by mpweiher 1y ago

https://tangled.sh/@rockorager.dev/lsr https://tangled.sh/@rockorager.dev/lsr

movomito 1y ago
Link doesn’t work
- Imustaskforhelp 1y ago
  Hm, well I have replied it to some other comment too but the link is working fine for me. Currently downloading zig to build it.
SillyUsername 1y ago
Love it. I'm trying to understand why all command line tools don't use io_uring. As an example, all my nvme's on usb 3.2 gen 2 only reach 740MB/s peak. If I use tools with aio or io_uring I get 1005MB/s. I know I may not be copying many files simultaneously every time, but the queue length strategies and the fewer locks also help I guess.
- superkuh 1y ago
  One reason is so that they work in all linux environments rather than just bleeding edge installs from the last couple years.
- tyingq 1y ago
  Probably historical preference for portability without a bunch of #ifdef means platform+version-specific stuff is very late to get adopted. Though, at this point, the benefit of portability across various posixy platforms is much lower.
  - Retr0id 1y ago
    Has anyone written an io_uring "polyfill" library with fallback to standard posix-y IO? It could presumably be done via background worker threads - at a perf cost.
    - vlovich123 1y ago
      Seems like a huge lift since io_uring is an ever growing set of interfaces that is encompassing more and more of the kernel surface area. Also, the problem tends to not necessarily be that the io_uring interface isn’t available at compile time but a) the version you distribute to has a kernel with it disabled or you don’t have permission to use it meaning you need to do LD_preload magic or use a framework b) the kernel you’re using supports some of the interfaces you’re trying to use but not all. Not sure how you solve that one without using a framework. But I agree. It would be cool if it was transparent, but this is actually what a bunch of io-uring runtimes do, using epoll as a fallback (eg in Rust monoio)
      - namibj 1y ago
        You can just ask io_uring what commands you have available to you. Though the way of the background thread should be readily available/usable by just indirectly calling the syscall (-helper) and replacing it with a futex-based handshake/wrapper. If you're not using the backend-ordering-imposing link bit, you could probably even use minor futex trickery to dispatch multiple background threads to snatch up from the submission queue in a "grab one at a time" fashion.
- never_inline 1y ago
  Poe's law hits again.
- elcapitan 1y ago
  iirc io_uring also had some pretty significant security issues early on (a couple of years ago). Those should be fixed by now, but that probably dampened adoption as well.
  - jeffbee 1y ago
    Not years ago. io_uring has been a continuous parade of security problems, including a high severity one that wasn't fixed until a few months ago. Many large organizations have patched it out of their kernels on safety basis, which is one of the reasons it suffers from poor adoption.
  - raesene9 1y ago
    Last I checked it's blocked by most container runtimes exactly because of the security problems, and Google blocked io_uring across all their services. I've not checked recently if that's still the case, but https://security.googleblog.com/2023/06/learnings-from-kctf-vrps-42-linux.html https://security.googleblog.com/2023/06/learnings-from-kctf-... has some background.
- tln 1y ago
  Thats a great speed boost. What tools are these?
- Thaxll 1y ago
  io_uring is a security nightmare.
  - pjc50 1y ago
    How so?
    - Thaxll 1y ago
      This is a good read on the topic: https://chomp.ie/Blog+Posts/Put+an+io_uring+on+it+-+Exploiting+the+Linux+Kernel https://chomp.ie/Blog+Posts/Put+an+io_uring+on+it+-+Exploiti...
    - sim7c00 1y ago
      you give process direct access to a piece of kernel memory. its a reason why there is separation. thats all.
      - wtallis 1y ago
        Most of the security concerns with io_uring that I've seen aren't related to the shared buffers at all but simply stem from the fact that io_uring is a mechanism to instruct the kernel to do stuff without making system calls, so security measures that focus on what system calls a process is allowed to do are ineffective.
      - loeg 1y ago
        This isn't the issue; it's relatively easy to safely share some ring buffers. The issue was/is that io_uring is rapidly growing the equivalent of ~all historical Linux syscall interfaces and sometimes comparable security measures were missed on the new interfaces. (Also, stuff like seccomp filters on syscalls are kind of meaningless for io_uring.)
      - duped 1y ago
        ...don't you supply the memory in the submission queue? or do you mean the queues themselves?
- fpoling 1y ago
  io_uring is the asynchronous interface and that requires to use even-based architecture to use it effectively. But many command-line tools are still written is a straightforward sequential style. If C would have async or similar mechanism to pretend doing async programming sequentially, it would be easier to port. But without that a very significant refactoring is necessary. Besides, io_uring is not yet stable and who knows may be in 10 years it will be replaced by yet another mechanism to take advantage of even newer hardware. So simply waiting for io_uring prove it is here to stay is very viable strategy. Besides in 10 years we may have tools/AI that will do the rewrite automatically...
  - mananaysiempre 1y ago
    > If C would have async or similar mechanism to pretend doing async programming sequentially, it would be easier to port. The *context() family of formerly-POSIX functions (clownishly deprecated as “use pthreads instead”) is essentially a full implementation of stackful coroutines. Even the arguable design botch of them preserving the signal mask (the reason why they aren’t the go-to option even on Linux) is theoretically fixable on the libc level without system calls, it’s just a lot of work and very few can be bothered to do signals well. As far as stackless coroutines, there’s a wide variety of libraries used in embedded systems and such (see the recent discussion[1] for some links), which are by necessity awkward enough that I don’t see any of them becoming broadly accepted. There were also a number of language extensions, among which I’d single out AC[2] (from the Barrelfish project) and CPC[3]. I’d love for, say, CPC to catch on, but it’s been over a decade now. [1] https://news.ycombinator.com/item?id=44546640 https://news.ycombinator.com/item?id=44546640 [2] https://users.soe.ucsc.edu/~abadi/Papers/acasync.pdf https://users.soe.ucsc.edu/~abadi/Papers/acasync.pdf [3] https://www.irif.fr/~jch/research/cpc-2012.pdf https://www.irif.fr/~jch/research/cpc-2012.pdf
  - ThePallas 1y ago
    As an example of pretending to do exactly that, this project (author: me) combines io_uring and ucontext.. https://github.com/pallas/ioucontext/ https://github.com/pallas/ioucontext/
- cesarb 1y ago
  > I'm trying to understand why all command line tools don't use io_uring. Because it's fairly new. The coreutils package which contains the ls command (and the three earlier packages which were merged to create it) is decades old; io_uring appeared much later. It will take time for the "shared ring buffer" style of system call to win over traditional synchronous system calls.
- Agingcoder 1y ago
  Iouring is very recent
fermuch 1y ago
The link isn't working for me. For those who were able to see it: does it improve anything by using that instead of what ls does now??
- Imustaskforhelp 1y ago
  Hm interesting, it worked for me.
- ta988 1y ago
  70% faster, but more importantly 35x times less syscalls.
  - eviks 1y ago
    Is there a noticeable benefit of this huge syscall reduction?
    - Imustaskforhelp 1y ago
      Yes I just checked it after installing strace strace -c ls gave me this 100.00 0.002709 13 198 5 total strace -c eza gave me this 100.00 0.006125 12 476 48 total strace -c lsr gave me this 100.00 0.001277 33 38 total So seeing the number of syscalls in the calls directory 198 : ls 476 : eza 33 : lsr A meaningful difference indeed!
      - richardwhiuk 1y ago
        That's just observing there is a difference, not explaining why that's a good thing.
        fpoling 1y ago
        syscalls are expensive and their relative latency compared with the rest of code only grow especially in view of mitigations against cache-related and other other hardware bugs.
ninkendo 1y ago
I wonder how it performs against an NFS server with lots of files, especially one over a kinda-crappy connection. Putting an unreliable network service behind blocking POSIX syscalls is one of the main reasons NFS is a terrible design choice (as can be seen by anyone who's tried to ctrl+c any app that's reading from a broken NFS folder), but I wonder if io_uring mitigates the bad parts somewhat.
- loeg 1y ago
  > as can be seen by anyone who's tried to ctrl+c any app that's reading from a broken NFS folder Theoretically "intr" mounts allowed signals to interrupt operations waiting on a hung remote server, but Linux removed the option long ago[1] (FreeBSD still supports it)[2]. "soft" might be the only workaround on Linux. [1]: https://man7.org/linux/man-pages/man5/nfs.5.html https://man7.org/linux/man-pages/man5/nfs.5.html [2]: https://man.freebsd.org/cgi/man.cgi?query=mount_nfs&sektion=8&apropos=0&manpath=FreeBSD+14.3-RELEASE+and+Ports https://man.freebsd.org/cgi/man.cgi?query=mount_nfs&sektion=...
- ape4 1y ago
  Samba too
- mprovost 1y ago
  The designers of NFS chose to make a distributed system emulate a highly consistent and available system (a hard drive), which was (and is) a reasonable tradeoff. It didn't require every existing tool, such as ls, to deal with things like the server rebooting while listing a directory. (The original NFS protocol is stateless, so clients can survive server reboots.) What does vi do when the server hosting the file you're editing stop responding? None of these tools have that kind of error handling. I don't know how io_uring solves this - does it return an error if the underlying NFS call times out? How long do you wait for a response before giving up and returning an error?
  - ninkendo 1y ago
    > The designers of NFS chose to make a distributed system emulate a highly consistent and available system (a hard drive), which was (and is) a reasonable tradeoff I don't agree that it was a reasonable tradeoff. Making an unreliable system emulate a reliable one is the very thing I find to be a bad idea. I don't think this is unique to NFS, it applies to any network filesystem you try to present as if it's a local one. > What does vi do when the server hosting the file you're editing stop responding? None of these tools have that kind of error handling. That's exactly why I don't think it's a good idea to just pretend a network connection is actually a local disk. Because tools aren't set up to handle issues with it being down. Contrast it with approaches where the client is aware of the network connection (like HTTP/GRPC/etc)... the client can decide for itself how long it should retry failed requests, whether it should bubble up failures to the caller, or work "offline" until it gets an opportunity to resync, etc. With NFS the syscall just hangs forever by default. Distributed systems are hard, and NFS (and other similar network filesystems) just pretend it isn't hard at all, which is great until something goes wrong, and then the abstraction leaks. (Also I didn't say io_uring solves this, but I'm curious as to whether its performance would be any better than blocking calls.)
Imustaskforhelp 1y ago
Really interesting, the difference is real though I would just hope that some better coloring support could be added because I have "eza --icons=always -1" command set as my ls and it looks really good, whereas when I use lsr -1, yes the fundamental thing is same, the difference is in the coloring. Yes lsr also colors the output but it doesn't know as many things as eza does For example .opus will show up as a music icon and with the right color (green-ish in my case?) in eza whereas it would be shown up as any normal file in lsr. Really no regrets though, its quite easy to patch I think but yes this is rock solid and really fast I must admit. Can you please create more such things but for cat and other system utilities too please? Also love that its using tangled.sh which is using atproto, kinda interesting too. I also like that its written in zig which imo feels way more easier for me to touch as a novice than rust (sry rustaceans)
- johnisgood 1y ago
  As for coloring support, I think the best way would be to implement LS_COLORS / dircolors. My GNU ls looks nice.
- iknowstuff 1y ago
  [flagged]
- hnlmorg 1y ago
  “bat” is a pretty good modern “cat” https://github.com/sharkdp/bat https://github.com/sharkdp/bat
  - Imustaskforhelp 1y ago
    So I just ran strace -c cat <file> and strace -c bat <file> Bat did 445 syscall Cat did 48 syscall Sure bat does beautify some things a lot but still I just wanted to tell this, I want something that can use io_uring for cat too I think, like what's the least number of syscalls that you can use for something like cat?
Bender 1y ago
I am curious what would happen if ls and other commands were replaced using io_uring and kernel.io_uring_disabled was set to 1. Would it fall back to an older behavior or would the ability to disable it be removed?
- rockorager 1y ago
  You would have to write your IO to have a fallback. The Ghostty project uses `io_uring`, but on kernels where it isn't available it falls back to an `epoll` model. That's all handled at the library level by libxev.
- yencabulator 1y ago
  I just realized that one could probably write a userspace io_uring emulator in a library that spawns a thread to read the ringbuffer and a worker pool of threads to do the blocking operations. You'd need to get the main software to make calls to your library instead of the io_uring syscalls, that's it; the app logic could remain the same. Then all the software wanting to use io_uring wouldn't need to write their low-level things twice.
  - Asmod4n 1y ago
    I'm about to start something like this targetting epoll, poll, dispatch_io and maybe kqueue this weekend.
neuroelectron 1y ago
There used to be lsring by Jens Axboe (author of io_uring), but it no longer exists. This is more extreme than abandoning the project. Perhaps there is some issue with using io_uring this way, perhaps vulnerabilities are exposed.
- arghwhat 1y ago
  > Perhaps there is some issue with using io_uring this way, perhaps vulnerabilities are exposed. ... no. It's just not interesting or particularly valuable to optimize ls, and Jens probably just used it as a demo and didn't want to keep it around.
  - neuroelectron 1y ago
    I'm sure there are uses in Bash scripts that could benefit from it but most people would use it directly in a compiled program, I suppose, if the performance was a reoccurring need.
  - neuroelectron 1y ago
    Explicit Vulnerabilities (Documented CVEs and Exploits) These are actual discovered vulnerabilities, typically assigned CVEs and often exploited in sandbox escapes or privilege escalations: 1. CVE-2021-3491 (Kernel 5.11+) Type: Privilege escalation Mechanism: Failure to check CAP_SYS_ADMIN before registering io_uring restrictions allowed unprivileged users to bypass sandboxing. Impact: Bypass of security policy mechanisms. 2. CVE-2022-29582 Type: UAF (Use-After-Free) Mechanism: io_uring allowed certain memory structures to be freed and reused improperly. Impact: Local privilege escalation. 3. CVE-2023-2598 Type: Race condition Mechanism: A race in the io_uring timeout code could lead to memory corruption. Impact: Arbitrary code execution or kernel crash. 4. CVE-2022-2602, CVE-2022-1116, etc. Type: UAF and out-of-bounds access Impact: Escalation from containers or sandboxed processes. 5. Exploit Tooling: Tools like io_uring_shock and custom kernel exploits often target io_uring in container escape scenarios (esp. with Docker or LXC). Implicit Vulnerabilities (Architectural and Latent Risks) These are not necessarily exploitable today, but reflect deeper systemic design risks or assumptions. 1. Shared Memory Abuse io_uring uses shared rings (memory-mapped via mmap) between kernel and user space. Risk: If ring buffer memory management has reference count bugs, attackers could force races, data corruption, or misuse stale pointers. 2. User-Controlled Kernel Pointers Some features allow user-specified buffers, SQEs, and CQEs to reference arbitrary memory (e.g. via IORING_OP_PROVIDE_BUFFERS, IORING_OP_MSG_RING). Risk: Incomplete validation could allow crafting fake kernel structures or triggering speculative attacks. 3. Speculative Execution & Side Channels Since io_uring relies on pre-submitted work queues and long-lived kernel threads, it opens timing side channels. Risk: Predictable scheduling or timing leaks, esp. combined with hardware speculation (Spectre-class). 4. Bypassing seccomp or AppArmor Filters io_uring operations can effectively batch or obscure syscall behavior. Example: A program restricted from calling sendmsg() directly might still use io_uring to perform similar actions. Risk: Policy enforcement tools become less effective, requiring explicit io_uring filtering. 5. Poor Auditability The batched and asynchronous nature makes logging or syscall audit trails incomplete or confusing. Risk: Harder for defenders or monitoring tools to track intent or detect misuse in real time. 6. Ring Reuse + Threaded Offload With IORING_SETUP_SQPOLL or IORING_SETUP_IOPOLL, I/O workers can run in kernel threads detached from user context. Risk: Desynchronized security context can lead to privileged operations escaping sandbox context (e.g., post-chroot but pre-fork). 7. File Descriptor Reuse and Lifecycle Mismatch Some operations in io_uring rely on fixed file descriptors or registered files. Race conditions with FD reuse or closing can cause inconsistencies. Risk: UAF, type confusion, or logic bombs triggered by kernel state confusion. Emerging Threat Vectors eBPF + io_uring Some exploits chain io_uring with eBPF to do arbitrary memory reads or writes. e.g., io_uring to perform controlled allocations, then eBPF to read or write memory. io_uring + userfaultfd Combining userfaultfd with io_uring allows very fine-grained control over page faults during I/O — great for fuzzing, also for exploit primitives.
rkangel 1y ago
This was more interesting for the tangled.sh platform it's hosted on. Wasn't aware of that!
- nikodunk 1y ago
  Same! Just signed up and will be following tangled and this repo. I like how tangled is built on atproto (bluesky).
  - seanw444 1y ago
    Is there any actual focus on ATProto as a decentralized protocol? So far it seems like its only purpose is building Bluesky as a centralized service, which I have no interest in at all.
    - Retr0id 1y ago
      Doesn't the existence of tangled answer your question?
      - seanw444 1y ago
        No, because when I go to sign up for the service, it asks for a Bluesky account name specifically. I don't care for yet another Facebook sign-in. I want something on par with email.
- dang 1y ago
  One past thread so far: Show HN: Tangled – Git collaboration platform built on atproto - https://news.ycombinator.com/item?id=43234544 https://news.ycombinator.com/item?id=43234544 - March 2025 (15 comments)
quibono 1y ago
Lovely, I might try doing this for some other "classic" utility! A bit off-topic too, but I'm new to Zig and curious. This here: ``` const allocator = sfb.get(); var cmd: Command = .{ .arena = allocator }; ``` means that all allocations need to be written with an allocator in mind? I.e. one has to pick an allocator per each memory allocation? Or is there a default one?
- kristoff_it 1y ago
  Allocator is an interface so you write library code only once, and then the caller decides which concrete implementation to use. There's cases where you do want to change your code based on the expectation that you will be provided a special kind of allocator (e.g. arenas), but that's a more niche thing and in any case it all comes together pretty well in practice.
- IggleSniggle 1y ago
  Caveat emptor, I don't write Zig but followed its development closely for awhile. A core design element of zig is that you shouldn't be stuck with one particular memory model. Zig encourages passing an allocator context around, where those allocators conform to a standardized interface. That means you could pass in different allocators with different performance characteristics at runtime. But yes, there is a default allocator, std.heap.page_allocator
  - hansvm 1y ago
    std.heap.smp_allocator You should basically only use the page allocator if you're writing another allocator.
  - SkiFire13 1y ago
    > you shouldn't be stuck with one particular memory model Nit: an allocator is not a "memory model", and I very much want the memory model to not change under my feet.
  - throwawaymaths 1y ago
    > Zig encourages passing an allocator context around, where those allocators conform to a standardized interface. in libraries. if youre just writing a final product it's totally fine to pick one and use it everywhere. > std.heap.page_allocator strongly disrecommend using this allocator as "default", it will take a trip to kernelland on each allocation.
danbruc 1y ago
Why does this require inventing lsr as an alternative to ls instead of making ls use io_uring? It seems pretty annoying to have to install replacements for the most basic command line tools. And especially in this case, where you do not even do it for additional features, just for getting the exact same thing done a bit faster.
- nailer 1y ago
  `ls` is in C, `lsr` is in Zig. The `lsr` programmer probably doesn't want to make new code in C.
  - loeg 1y ago
    In addition, the author might not want to sign away their rights to the FSF.
    - deleted 1y ago
      [deleted]
    - andrepd 1y ago
      What on earth are you talking about? Why would this be the case?
      - scott_w 1y ago
        Depending on the implementation (and I don't know which `ls` is being referred to), modifying `ls` might mean modifying an FSF project which require copyright assignment as a condition of patch submissions.
        leni536 1y ago
        That's only the case if the author would want to upstream their changes. If they wanted to only fork ls then they would only be required to comply with the license, without assigning copyright over.
adgjlsfhk1 1y ago
It's a shame to see uutils doing so poorly here. I feel like they're our best hope for an organization to drive this sort of core modernization forward, but 2x slower than GNU isn't a good start.
- sureglymop 1y ago
  I mean not only that but they're also licensed differently. I'd prefer strong copyleft for the most fundamental building blocks. Other than that it's a decently interesting project.
ReDress 1y ago
I've been playing around with io_uring for a while. Still, I am yet to come across a some tests that simulate typical real life application workload. I heard of fio but are yet to check how exactly it works and whether it might be possible to simulate real life application workload with it.
- izabera 1y ago
  what a "real life application workload" looks like is entirely dependent on your use case, but fio is very widely used in the storage industry it's a good first approximation to test the cartesian product of - sequential/random - reads/writes - in arbitrary sizes - with arbitrarily many workers - with many different backends to perform such i/o including io_uring and its reporting is solid and thorough implementing the same for your specific workload is often not trivial at all
the8472 1y ago
io_uring doesn't support getdents though. so the primary benefit is bulk statting (ls -l). It'd be nice if we could have a getdents in flight while processing the results of the previous one.
- loeg 1y ago
  POSIX adopting NFS' "readdirplus" operation (getdents + stat) could negate some of the benefit towards io_uring, too.
  - the8472 1y ago
    but then someone wants statx...
    - loeg 1y ago
      Yeah. To be clear I don't think Linux or Posix will ever adopt readdirplus as an API.
tln 1y ago
The times seem sublinear, 10k files is less than 10x 1k files. I remember getting in to a situation during the ext2 and spinning rust days where production directories had 500k files. ls processes were slow enough to overload everything. ls -F saved me there. And filesystems got a lot better at lots of files. What filesystem was used here? It's interesting how well busybox fares, it's written for size not speed iirc?
- SkiFire13 1y ago
  > The times seem sublinear, 10k files is less than 10x 1k files Two points are not enough to say it's sublinear. It might very well be some constant factor that becomes less and less important the bigger the linear factor becomes. Or in other words 10000n+C < 10000(n+C)
  - tln 1y ago
    The article has data points for n=10,100,1000,10000. Taking (n=10,000 - n=10)/(n=1,000 - n=10) would eliminate the constant factor and we'd expect about 10.09x higher times for a linear algorithm. But for lsr, it's 9.34. The other tools have factors close to 10.09 or higher. Since ls has to sort it's output (unless -F is specified) I'd not be too surprised with a little superlinearity. https://docs.google.com/spreadsheets/d/1EAYua3B3UeTGBtAejPw2PfVK__Fd_x2PVe0xW20cFAU/edit?gid=119533988#gid=119533988 https://docs.google.com/spreadsheets/d/1EAYua3B3UeTGBtAejPw2...
- otterley 1y ago
  Ext2 never got better with large directories even with SSDs (this includes up to ext4). The benchmarks don’t include the filesystem type, which is actually extremely important when it comes to the performance of reading directories.
maplant 1y ago
This seems more interesting as demonstration of the amortized performance increase you'd expect from using io_uring, or as a tutorial for using it. I don't understand why I'd switch from using something like eza. If I'm listing 10,000 files the difference is between 40ms and 20ms. I absolutely would not notice that for a single invocation of the command.
- 0x000xca0xfe 1y ago
  Well I have a directory with a couple million JSON files and ls/du take minutes. Most of the coreutils are not fast enough to actually utilize modern SSDs.
  - otterley 1y ago
    What’s the filesystem type? Ext4 suffers terrible lookup performance with large directories, while xfs absolutely flies.
    - 0x000xca0xfe 1y ago
      Yup, default ext4 and most files are <4KB, so it's extra bad. Thanks for the comment, didn't know that!
- rockorager 1y ago
  Yeah, I wrote this as a fun little experiment to learn more io_uring usage. The practical savings of using this are tiny, maybe 5 seconds over your entire life. That wasn't the point haha
  - JuettnerDistrib 1y ago
    I'd be curious to know if this helps on supercomputers, which are notorious for frequently hanging for a few seconds on an ls -l.
    - mrlongroots 1y ago
api 1y ago
Why isn’t it possible — or is it — to make libc just use uring instead of syscall? Yes I know uring is an async interface, but it’s trivial to implement sync behavior on top of a single chain of async send-wait pairs, like doing a simple single threaded “conversational” implementation of a network protocol. It wouldn’t make a difference in most individual cases but overall I wonder how big a global speed boost you’d get by removing a ton of syscalls? Or am I failing to understand something about the performance nuances here?
- ninkendo 1y ago
  In order to make this work, libc would have to: - Start some sort of async executor thread to service the io_uring requests/responses - Make it so every call to "normal" syscalls causes the calling thread to sleep until the result is available (that's 1 syscall) - When the executor thread gets a result, have it wake up the original thread (that's another syscall) So you're basically turning 1 syscall into 2 in order to emulate the legacy syscalls. io_uring only makes sense if you're already async. Emulating sync on top of async is nearly always a terrible idea.
  - wtallis 1y ago
    You don't need to start spawning new threads to use io_uring as a backend for synchronous IO APIs. You just need to set up the rings once, then when the program does an fwrite or whatever, that gets implemented as sending a submission queue entry followed by a single io_uring_enter syscall that informs the kernel there's something in the submission queue, and using the arguments indicating that the calling process wants to block until there's something in the completion queue.
    - ninkendo 1y ago
      > using the arguments indicating the calling process wants to block Nice to know io_uring has facilities for backwards compatibility with blocking code here. But yeah, that's still a syscall, and given that the whole benefit of io_uring is in avoiding (or at least, coalescing) syscalls, I doubt having libc "just" use io_uring is going to give any tangible benefit.
rockorager 1y ago
Author of the project here! I have a little write up on this here: https://rockorager.dev/log/lsr-ls-but-with-io-uring https://rockorager.dev/log/lsr-ls-but-with-io-uring
- jeffbee 1y ago
  How much of the speedup over GNU ls is due to lacking localization features? Your results table is pretty much consistent with my local observations: in a dir with 13k files, `ls -al` needs 33ms. But 25% of that time is spent by libc in `strcoll`. Under `LC_ALL=C` it takes just 27ms, which is getting closer to the time of your program.
  - rockorager 1y ago
    I didn't include `busybox` in my initial table, so it isn't on the blog post but the repo has the data...but I am 99% sure busybox does not have locale support, so I think GNU ls without locale support would probably be closer to busybox. Locales also bring in a lot more complicated sorting - so that could be a factor also.
- tavianator 1y ago
  My bfs project also uses io_uring: https://github.com/tavianator/bfs/blob/main/src/ioq.c https://github.com/tavianator/bfs/blob/main/src/ioq.c I'm curious how lsr compares to bfs -ls for example. bfs only uses io_uring when multiple threads are enabled, but maybe it's worth using it even for bfs -j1
  - rockorager 1y ago
    Oh that's cool. `find` is another tool I thought could benefit from io_uring like `ls`. I think it's definitely worth enabling io_uring for single threaded applications for the batching benefit. The kernel will still spin up a thread pool to get the work done concurrently, but you don't have to manage that in your codebase.
    - 1y ago
jasonjmcghee 1y ago
I find it funny that there are icons for .mjs and .cjs file extensions but not .c, .h, .sh
swiftcoder 1y ago
Kind of fascinating that slashing syscalls by ~35x (versus the `ls -la` benchmark) is "only" worth a 2x speedup
- bogwog 1y ago
  I vaguely remember some benchmark I read a while back for some other io_uring project, and it suggested that io_uring syscalls are more expensive than whatever the other syscalls were that it was being used to replace. It's still a big improvement, even if not as big as you'd hope. I wish I could remember the post, but I've had that impression in the back of my mind ever since.
- Galanwe 1y ago
  These syscalls are mostly through VDSO, so not very costly
  - wahern 1y ago
    The only VDSO-capable calls are clock_gettime, getcpu, getrandom, gettimeofday, and time. (Other architectures have some more, mostly related to signals and CPU cache flushing.)
- Too 1y ago
  Can someone try this in a virtual machine? I’d imagine syscalls there to be more expensive.
mnw21cam 1y ago
Love the idea and execution, don't love the misplaced apo'strophe's.
- rockorager 1y ago
  Oh no - where at?
  - nbf_1995 1y ago
    Technically, the first, third, and fifth occurrence of "it's" should be "its". The dog chased its tail. I didn't notice when I read the article though. The original commenter is being pedantic.
  - teo_zero 1y ago
    And since you're asking, beware of "let's" too: the third one is wrong!
pvtmert 1y ago
> I have no idea what lsd is doing. I haven’t read the source code, but from viewing it’s strace, it is calling clock_gettime around 5 times per file. Why? I don’t know. Maybe it’s doing internal timing of steps along the way? Maybe calculating "X minutes/hours/days/weeks ago" thing for each timestamp? (access, create, modify, ...). Could just be an old artifact of another library function...
- namibj 1y ago
  This shouldn't be an actual syscall these days; it should be handled by vDSO (`man 7 vDSO`). Maybe zig doesn't use that, though.
buybackoff 1y ago
A little offtop, but do you know a number in usecs that io_uring can save on enterprise grade servers, with 10G NICs, for socket latency overheads vs LD_PRELOAD when hardware supports that? Let's say it's Mellanox 4 or 5. My understanding is that each gives around 10us savings, maybe less. Based on some benchmarking, which was not focused on any of those explicitly but had some imprecise experiments. It also looks like they do not add up. Do you have a number based on real experience?
_ache_ 1y ago
Why doesn't it scale ? I'm really interested in n=100.000 and n=1.000.000. eza could catch up with lsr. Why is n=1.000 so slow with lsr ? Does io_uring add so much overhead ?
Lockal 1y ago
A relevant task in uutils https://github.com/uutils/coreutils/issues/4362 https://github.com/uutils/coreutils/issues/4362 describes how io_uring universally helps in almost any disk operation util.
michaelcampbell 1y ago
This is awesome for learning and a funzies app. Would be nice to see it moved into some mainstream distro, to be honest. That said, in over 35 years of using `ls`, never once have I bemoaned the speed of it.
- flemhans 1y ago
  Using ls in a dir with millions of files may stall a good while.
  - michaelcampbell 1y ago
    Sure; I guess scaling personal things to Google scale just isn't something I've had to deal with as a pleb software dev/user.
snapplebobapple 1y ago
This is cool. I really miss the eza coloring of the file permissions/ownership/size/dates. I've installed it anyway to try when eza -l is eating it because of a large number of files in the directory. Side note: would something like this make ncdu faster? I run into large file counts = slow using that a lot more than the ls command.

351 points by mpweiher 1y ago

https://tangled.sh/@rockorager.dev/lsr https://tangled.sh/@rockorager.dev/lsr

movomito 1y ago

Link doesn’t work

Imustaskforhelp 1y ago
Hm, well I have replied it to some other comment too but the link is working fine for me. Currently downloading zig to build it.

SillyUsername 1y ago

Love it. I'm trying to understand why all command line tools don't use io_uring. As an example, all my nvme's on usb 3.2 gen 2 only reach 740MB/s peak. If I use tools with aio or io_uring I get 1005MB/s. I know I may not be copying many files simultaneously every time, but the queue length strategies and the fewer locks also help I guess.

superkuh 1y ago
One reason is so that they work in all linux environments rather than just bleeding edge installs from the last couple years.
tyingq 1y ago
Probably historical preference for portability without a bunch of #ifdef means platform+version-specific stuff is very late to get adopted. Though, at this point, the benefit of portability across various posixy platforms is much lower.
- Retr0id 1y ago
  Has anyone written an io_uring "polyfill" library with fallback to standard posix-y IO? It could presumably be done via background worker threads - at a perf cost.
  - vlovich123 1y ago
    Seems like a huge lift since io_uring is an ever growing set of interfaces that is encompassing more and more of the kernel surface area. Also, the problem tends to not necessarily be that the io_uring interface isn’t available at compile time but a) the version you distribute to has a kernel with it disabled or you don’t have permission to use it meaning you need to do LD_preload magic or use a framework b) the kernel you’re using supports some of the interfaces you’re trying to use but not all. Not sure how you solve that one without using a framework. But I agree. It would be cool if it was transparent, but this is actually what a bunch of io-uring runtimes do, using epoll as a fallback (eg in Rust monoio)
    - namibj 1y ago
      You can just ask io_uring what commands you have available to you. Though the way of the background thread should be readily available/usable by just indirectly calling the syscall (-helper) and replacing it with a futex-based handshake/wrapper. If you're not using the backend-ordering-imposing link bit, you could probably even use minor futex trickery to dispatch multiple background threads to snatch up from the submission queue in a "grab one at a time" fashion.
never_inline 1y ago
Poe's law hits again.
elcapitan 1y ago
iirc io_uring also had some pretty significant security issues early on (a couple of years ago). Those should be fixed by now, but that probably dampened adoption as well.
- jeffbee 1y ago
  Not years ago. io_uring has been a continuous parade of security problems, including a high severity one that wasn't fixed until a few months ago. Many large organizations have patched it out of their kernels on safety basis, which is one of the reasons it suffers from poor adoption.
- raesene9 1y ago
  Last I checked it's blocked by most container runtimes exactly because of the security problems, and Google blocked io_uring across all their services. I've not checked recently if that's still the case, but https://security.googleblog.com/2023/06/learnings-from-kctf-vrps-42-linux.html https://security.googleblog.com/2023/06/learnings-from-kctf-... has some background.
tln 1y ago
Thats a great speed boost. What tools are these?
Thaxll 1y ago
io_uring is a security nightmare.
- pjc50 1y ago
  How so?
  - Thaxll 1y ago
    This is a good read on the topic: https://chomp.ie/Blog+Posts/Put+an+io_uring+on+it+-+Exploiting+the+Linux+Kernel https://chomp.ie/Blog+Posts/Put+an+io_uring+on+it+-+Exploiti...
  - sim7c00 1y ago
    you give process direct access to a piece of kernel memory. its a reason why there is separation. thats all.
    - wtallis 1y ago
      Most of the security concerns with io_uring that I've seen aren't related to the shared buffers at all but simply stem from the fact that io_uring is a mechanism to instruct the kernel to do stuff without making system calls, so security measures that focus on what system calls a process is allowed to do are ineffective.
    - loeg 1y ago
      This isn't the issue; it's relatively easy to safely share some ring buffers. The issue was/is that io_uring is rapidly growing the equivalent of ~all historical Linux syscall interfaces and sometimes comparable security measures were missed on the new interfaces. (Also, stuff like seccomp filters on syscalls are kind of meaningless for io_uring.)
    - duped 1y ago
      ...don't you supply the memory in the submission queue? or do you mean the queues themselves?
fpoling 1y ago
io_uring is the asynchronous interface and that requires to use even-based architecture to use it effectively. But many command-line tools are still written is a straightforward sequential style. If C would have async or similar mechanism to pretend doing async programming sequentially, it would be easier to port. But without that a very significant refactoring is necessary. Besides, io_uring is not yet stable and who knows may be in 10 years it will be replaced by yet another mechanism to take advantage of even newer hardware. So simply waiting for io_uring prove it is here to stay is very viable strategy. Besides in 10 years we may have tools/AI that will do the rewrite automatically...
- mananaysiempre 1y ago
  > If C would have async or similar mechanism to pretend doing async programming sequentially, it would be easier to port. The *context() family of formerly-POSIX functions (clownishly deprecated as “use pthreads instead”) is essentially a full implementation of stackful coroutines. Even the arguable design botch of them preserving the signal mask (the reason why they aren’t the go-to option even on Linux) is theoretically fixable on the libc level without system calls, it’s just a lot of work and very few can be bothered to do signals well. As far as stackless coroutines, there’s a wide variety of libraries used in embedded systems and such (see the recent discussion[1] for some links), which are by necessity awkward enough that I don’t see any of them becoming broadly accepted. There were also a number of language extensions, among which I’d single out AC[2] (from the Barrelfish project) and CPC[3]. I’d love for, say, CPC to catch on, but it’s been over a decade now. [1] https://news.ycombinator.com/item?id=44546640 https://news.ycombinator.com/item?id=44546640 [2] https://users.soe.ucsc.edu/~abadi/Papers/acasync.pdf https://users.soe.ucsc.edu/~abadi/Papers/acasync.pdf [3] https://www.irif.fr/~jch/research/cpc-2012.pdf https://www.irif.fr/~jch/research/cpc-2012.pdf
- ThePallas 1y ago
  As an example of pretending to do exactly that, this project (author: me) combines io_uring and ucontext.. https://github.com/pallas/ioucontext/ https://github.com/pallas/ioucontext/
cesarb 1y ago
> I'm trying to understand why all command line tools don't use io_uring. Because it's fairly new. The coreutils package which contains the ls command (and the three earlier packages which were merged to create it) is decades old; io_uring appeared much later. It will take time for the "shared ring buffer" style of system call to win over traditional synchronous system calls.
Agingcoder 1y ago
Iouring is very recent

fermuch 1y ago

The link isn't working for me. For those who were able to see it: does it improve anything by using that instead of what ls does now??

Imustaskforhelp 1y ago
Hm interesting, it worked for me.
ta988 1y ago
70% faster, but more importantly 35x times less syscalls.
- eviks 1y ago
  Is there a noticeable benefit of this huge syscall reduction?
  - Imustaskforhelp 1y ago
    Yes I just checked it after installing strace strace -c ls gave me this 100.00 0.002709 13 198 5 total strace -c eza gave me this 100.00 0.006125 12 476 48 total strace -c lsr gave me this 100.00 0.001277 33 38 total So seeing the number of syscalls in the calls directory 198 : ls 476 : eza 33 : lsr A meaningful difference indeed!
    - richardwhiuk 1y ago
      That's just observing there is a difference, not explaining why that's a good thing.
      - fpoling 1y ago
        syscalls are expensive and their relative latency compared with the rest of code only grow especially in view of mitigations against cache-related and other other hardware bugs.

ninkendo 1y ago

I wonder how it performs against an NFS server with lots of files, especially one over a kinda-crappy connection. Putting an unreliable network service behind blocking POSIX syscalls is one of the main reasons NFS is a terrible design choice (as can be seen by anyone who's tried to ctrl+c any app that's reading from a broken NFS folder), but I wonder if io_uring mitigates the bad parts somewhat.

loeg 1y ago
> as can be seen by anyone who's tried to ctrl+c any app that's reading from a broken NFS folder Theoretically "intr" mounts allowed signals to interrupt operations waiting on a hung remote server, but Linux removed the option long ago[1] (FreeBSD still supports it)[2]. "soft" might be the only workaround on Linux. [1]: https://man7.org/linux/man-pages/man5/nfs.5.html https://man7.org/linux/man-pages/man5/nfs.5.html [2]: https://man.freebsd.org/cgi/man.cgi?query=mount_nfs&sektion=8&apropos=0&manpath=FreeBSD+14.3-RELEASE+and+Ports https://man.freebsd.org/cgi/man.cgi?query=mount_nfs&sektion=...
ape4 1y ago
Samba too
mprovost 1y ago
The designers of NFS chose to make a distributed system emulate a highly consistent and available system (a hard drive), which was (and is) a reasonable tradeoff. It didn't require every existing tool, such as ls, to deal with things like the server rebooting while listing a directory. (The original NFS protocol is stateless, so clients can survive server reboots.) What does vi do when the server hosting the file you're editing stop responding? None of these tools have that kind of error handling. I don't know how io_uring solves this - does it return an error if the underlying NFS call times out? How long do you wait for a response before giving up and returning an error?
- ninkendo 1y ago
  > The designers of NFS chose to make a distributed system emulate a highly consistent and available system (a hard drive), which was (and is) a reasonable tradeoff I don't agree that it was a reasonable tradeoff. Making an unreliable system emulate a reliable one is the very thing I find to be a bad idea. I don't think this is unique to NFS, it applies to any network filesystem you try to present as if it's a local one. > What does vi do when the server hosting the file you're editing stop responding? None of these tools have that kind of error handling. That's exactly why I don't think it's a good idea to just pretend a network connection is actually a local disk. Because tools aren't set up to handle issues with it being down. Contrast it with approaches where the client is aware of the network connection (like HTTP/GRPC/etc)... the client can decide for itself how long it should retry failed requests, whether it should bubble up failures to the caller, or work "offline" until it gets an opportunity to resync, etc. With NFS the syscall just hangs forever by default. Distributed systems are hard, and NFS (and other similar network filesystems) just pretend it isn't hard at all, which is great until something goes wrong, and then the abstraction leaks. (Also I didn't say io_uring solves this, but I'm curious as to whether its performance would be any better than blocking calls.)

Imustaskforhelp 1y ago

Really interesting, the difference is real though I would just hope that some better coloring support could be added because I have "eza --icons=always -1" command set as my ls and it looks really good, whereas when I use lsr -1, yes the fundamental thing is same, the difference is in the coloring. Yes lsr also colors the output but it doesn't know as many things as eza does For example .opus will show up as a music icon and with the right color (green-ish in my case?) in eza whereas it would be shown up as any normal file in lsr. Really no regrets though, its quite easy to patch I think but yes this is rock solid and really fast I must admit. Can you please create more such things but for cat and other system utilities too please? Also love that its using tangled.sh which is using atproto, kinda interesting too. I also like that its written in zig which imo feels way more easier for me to touch as a novice than rust (sry rustaceans)

johnisgood 1y ago
As for coloring support, I think the best way would be to implement LS_COLORS / dircolors. My GNU ls looks nice.
iknowstuff 1y ago
[flagged]
hnlmorg 1y ago
“bat” is a pretty good modern “cat” https://github.com/sharkdp/bat https://github.com/sharkdp/bat
- Imustaskforhelp 1y ago
  So I just ran strace -c cat <file> and strace -c bat <file> Bat did 445 syscall Cat did 48 syscall Sure bat does beautify some things a lot but still I just wanted to tell this, I want something that can use io_uring for cat too I think, like what's the least number of syscalls that you can use for something like cat?

Bender 1y ago

I am curious what would happen if ls and other commands were replaced using io_uring and kernel.io_uring_disabled was set to 1. Would it fall back to an older behavior or would the ability to disable it be removed?

rockorager 1y ago
You would have to write your IO to have a fallback. The Ghostty project uses `io_uring`, but on kernels where it isn't available it falls back to an `epoll` model. That's all handled at the library level by libxev.
yencabulator 1y ago
I just realized that one could probably write a userspace io_uring emulator in a library that spawns a thread to read the ringbuffer and a worker pool of threads to do the blocking operations. You'd need to get the main software to make calls to your library instead of the io_uring syscalls, that's it; the app logic could remain the same. Then all the software wanting to use io_uring wouldn't need to write their low-level things twice.
- Asmod4n 1y ago
  I'm about to start something like this targetting epoll, poll, dispatch_io and maybe kqueue this weekend.

neuroelectron 1y ago

There used to be lsring by Jens Axboe (author of io_uring), but it no longer exists. This is more extreme than abandoning the project. Perhaps there is some issue with using io_uring this way, perhaps vulnerabilities are exposed.

arghwhat 1y ago
> Perhaps there is some issue with using io_uring this way, perhaps vulnerabilities are exposed. ... no. It's just not interesting or particularly valuable to optimize ls, and Jens probably just used it as a demo and didn't want to keep it around.
- neuroelectron 1y ago
  I'm sure there are uses in Bash scripts that could benefit from it but most people would use it directly in a compiled program, I suppose, if the performance was a reoccurring need.
- neuroelectron 1y ago
  Explicit Vulnerabilities (Documented CVEs and Exploits) These are actual discovered vulnerabilities, typically assigned CVEs and often exploited in sandbox escapes or privilege escalations: 1. CVE-2021-3491 (Kernel 5.11+) Type: Privilege escalation Mechanism: Failure to check CAP_SYS_ADMIN before registering io_uring restrictions allowed unprivileged users to bypass sandboxing. Impact: Bypass of security policy mechanisms. 2. CVE-2022-29582 Type: UAF (Use-After-Free) Mechanism: io_uring allowed certain memory structures to be freed and reused improperly. Impact: Local privilege escalation. 3. CVE-2023-2598 Type: Race condition Mechanism: A race in the io_uring timeout code could lead to memory corruption. Impact: Arbitrary code execution or kernel crash. 4. CVE-2022-2602, CVE-2022-1116, etc. Type: UAF and out-of-bounds access Impact: Escalation from containers or sandboxed processes. 5. Exploit Tooling: Tools like io_uring_shock and custom kernel exploits often target io_uring in container escape scenarios (esp. with Docker or LXC). Implicit Vulnerabilities (Architectural and Latent Risks) These are not necessarily exploitable today, but reflect deeper systemic design risks or assumptions. 1. Shared Memory Abuse io_uring uses shared rings (memory-mapped via mmap) between kernel and user space. Risk: If ring buffer memory management has reference count bugs, attackers could force races, data corruption, or misuse stale pointers. 2. User-Controlled Kernel Pointers Some features allow user-specified buffers, SQEs, and CQEs to reference arbitrary memory (e.g. via IORING_OP_PROVIDE_BUFFERS, IORING_OP_MSG_RING). Risk: Incomplete validation could allow crafting fake kernel structures or triggering speculative attacks. 3. Speculative Execution & Side Channels Since io_uring relies on pre-submitted work queues and long-lived kernel threads, it opens timing side channels. Risk: Predictable scheduling or timing leaks, esp. combined with hardware speculation (Spectre-class). 4. Bypassing seccomp or AppArmor Filters io_uring operations can effectively batch or obscure syscall behavior. Example: A program restricted from calling sendmsg() directly might still use io_uring to perform similar actions. Risk: Policy enforcement tools become less effective, requiring explicit io_uring filtering. 5. Poor Auditability The batched and asynchronous nature makes logging or syscall audit trails incomplete or confusing. Risk: Harder for defenders or monitoring tools to track intent or detect misuse in real time. 6. Ring Reuse + Threaded Offload With IORING_SETUP_SQPOLL or IORING_SETUP_IOPOLL, I/O workers can run in kernel threads detached from user context. Risk: Desynchronized security context can lead to privileged operations escaping sandbox context (e.g., post-chroot but pre-fork). 7. File Descriptor Reuse and Lifecycle Mismatch Some operations in io_uring rely on fixed file descriptors or registered files. Race conditions with FD reuse or closing can cause inconsistencies. Risk: UAF, type confusion, or logic bombs triggered by kernel state confusion. Emerging Threat Vectors eBPF + io_uring Some exploits chain io_uring with eBPF to do arbitrary memory reads or writes. e.g., io_uring to perform controlled allocations, then eBPF to read or write memory. io_uring + userfaultfd Combining userfaultfd with io_uring allows very fine-grained control over page faults during I/O — great for fuzzing, also for exploit primitives.

rkangel 1y ago

This was more interesting for the tangled.sh platform it's hosted on. Wasn't aware of that!

nikodunk 1y ago
Same! Just signed up and will be following tangled and this repo. I like how tangled is built on atproto (bluesky).
- seanw444 1y ago
  Is there any actual focus on ATProto as a decentralized protocol? So far it seems like its only purpose is building Bluesky as a centralized service, which I have no interest in at all.
  - Retr0id 1y ago
    Doesn't the existence of tangled answer your question?
    - seanw444 1y ago
      No, because when I go to sign up for the service, it asks for a Bluesky account name specifically. I don't care for yet another Facebook sign-in. I want something on par with email.
dang 1y ago
One past thread so far: Show HN: Tangled – Git collaboration platform built on atproto - https://news.ycombinator.com/item?id=43234544 https://news.ycombinator.com/item?id=43234544 - March 2025 (15 comments)

quibono 1y ago

Lovely, I might try doing this for some other "classic" utility! A bit off-topic too, but I'm new to Zig and curious. This here: ``` const allocator = sfb.get(); var cmd: Command = .{ .arena = allocator }; ``` means that all allocations need to be written with an allocator in mind? I.e. one has to pick an allocator per each memory allocation? Or is there a default one?

kristoff_it 1y ago
Allocator is an interface so you write library code only once, and then the caller decides which concrete implementation to use. There's cases where you do want to change your code based on the expectation that you will be provided a special kind of allocator (e.g. arenas), but that's a more niche thing and in any case it all comes together pretty well in practice.
IggleSniggle 1y ago
Caveat emptor, I don't write Zig but followed its development closely for awhile. A core design element of zig is that you shouldn't be stuck with one particular memory model. Zig encourages passing an allocator context around, where those allocators conform to a standardized interface. That means you could pass in different allocators with different performance characteristics at runtime. But yes, there is a default allocator, std.heap.page_allocator
- hansvm 1y ago
  std.heap.smp_allocator You should basically only use the page allocator if you're writing another allocator.
- SkiFire13 1y ago
  > you shouldn't be stuck with one particular memory model Nit: an allocator is not a "memory model", and I very much want the memory model to not change under my feet.
- throwawaymaths 1y ago
  > Zig encourages passing an allocator context around, where those allocators conform to a standardized interface. in libraries. if youre just writing a final product it's totally fine to pick one and use it everywhere. > std.heap.page_allocator strongly disrecommend using this allocator as "default", it will take a trip to kernelland on each allocation.

danbruc 1y ago

Why does this require inventing lsr as an alternative to ls instead of making ls use io_uring? It seems pretty annoying to have to install replacements for the most basic command line tools. And especially in this case, where you do not even do it for additional features, just for getting the exact same thing done a bit faster.

nailer 1y ago
`ls` is in C, `lsr` is in Zig. The `lsr` programmer probably doesn't want to make new code in C.
- loeg 1y ago
  In addition, the author might not want to sign away their rights to the FSF.
  - deleted 1y ago
    [deleted]
  - andrepd 1y ago
    What on earth are you talking about? Why would this be the case?
    - scott_w 1y ago
      Depending on the implementation (and I don't know which `ls` is being referred to), modifying `ls` might mean modifying an FSF project which require copyright assignment as a condition of patch submissions.
      - leni536 1y ago
        That's only the case if the author would want to upstream their changes. If they wanted to only fork ls then they would only be required to comply with the license, without assigning copyright over.

adgjlsfhk1 1y ago

It's a shame to see uutils doing so poorly here. I feel like they're our best hope for an organization to drive this sort of core modernization forward, but 2x slower than GNU isn't a good start.

sureglymop 1y ago
I mean not only that but they're also licensed differently. I'd prefer strong copyleft for the most fundamental building blocks. Other than that it's a decently interesting project.

ReDress 1y ago

I've been playing around with io_uring for a while. Still, I am yet to come across a some tests that simulate typical real life application workload. I heard of fio but are yet to check how exactly it works and whether it might be possible to simulate real life application workload with it.

izabera 1y ago
what a "real life application workload" looks like is entirely dependent on your use case, but fio is very widely used in the storage industry it's a good first approximation to test the cartesian product of - sequential/random - reads/writes - in arbitrary sizes - with arbitrarily many workers - with many different backends to perform such i/o including io_uring and its reporting is solid and thorough implementing the same for your specific workload is often not trivial at all

the8472 1y ago

io_uring doesn't support getdents though. so the primary benefit is bulk statting (ls -l). It'd be nice if we could have a getdents in flight while processing the results of the previous one.

loeg 1y ago
POSIX adopting NFS' "readdirplus" operation (getdents + stat) could negate some of the benefit towards io_uring, too.
- the8472 1y ago
  but then someone wants statx...
  - loeg 1y ago
    Yeah. To be clear I don't think Linux or Posix will ever adopt readdirplus as an API.

tln 1y ago

The times seem sublinear, 10k files is less than 10x 1k files. I remember getting in to a situation during the ext2 and spinning rust days where production directories had 500k files. ls processes were slow enough to overload everything. ls -F saved me there. And filesystems got a lot better at lots of files. What filesystem was used here? It's interesting how well busybox fares, it's written for size not speed iirc?

SkiFire13 1y ago
> The times seem sublinear, 10k files is less than 10x 1k files Two points are not enough to say it's sublinear. It might very well be some constant factor that becomes less and less important the bigger the linear factor becomes. Or in other words 10000n+C < 10000(n+C)
- tln 1y ago
  The article has data points for n=10,100,1000,10000. Taking (n=10,000 - n=10)/(n=1,000 - n=10) would eliminate the constant factor and we'd expect about 10.09x higher times for a linear algorithm. But for lsr, it's 9.34. The other tools have factors close to 10.09 or higher. Since ls has to sort it's output (unless -F is specified) I'd not be too surprised with a little superlinearity. https://docs.google.com/spreadsheets/d/1EAYua3B3UeTGBtAejPw2PfVK__Fd_x2PVe0xW20cFAU/edit?gid=119533988#gid=119533988 https://docs.google.com/spreadsheets/d/1EAYua3B3UeTGBtAejPw2...
otterley 1y ago
Ext2 never got better with large directories even with SSDs (this includes up to ext4). The benchmarks don’t include the filesystem type, which is actually extremely important when it comes to the performance of reading directories.

maplant 1y ago

This seems more interesting as demonstration of the amortized performance increase you'd expect from using io_uring, or as a tutorial for using it. I don't understand why I'd switch from using something like eza. If I'm listing 10,000 files the difference is between 40ms and 20ms. I absolutely would not notice that for a single invocation of the command.

0x000xca0xfe 1y ago
Well I have a directory with a couple million JSON files and ls/du take minutes. Most of the coreutils are not fast enough to actually utilize modern SSDs.
- otterley 1y ago
  What’s the filesystem type? Ext4 suffers terrible lookup performance with large directories, while xfs absolutely flies.
  - 0x000xca0xfe 1y ago
    Yup, default ext4 and most files are <4KB, so it's extra bad. Thanks for the comment, didn't know that!
rockorager 1y ago
Yeah, I wrote this as a fun little experiment to learn more io_uring usage. The practical savings of using this are tiny, maybe 5 seconds over your entire life. That wasn't the point haha
- JuettnerDistrib 1y ago
  I'd be curious to know if this helps on supercomputers, which are notorious for frequently hanging for a few seconds on an ls -l.
  - mrlongroots 1y ago

api 1y ago

Why isn’t it possible — or is it — to make libc just use uring instead of syscall? Yes I know uring is an async interface, but it’s trivial to implement sync behavior on top of a single chain of async send-wait pairs, like doing a simple single threaded “conversational” implementation of a network protocol. It wouldn’t make a difference in most individual cases but overall I wonder how big a global speed boost you’d get by removing a ton of syscalls? Or am I failing to understand something about the performance nuances here?

ninkendo 1y ago
In order to make this work, libc would have to: - Start some sort of async executor thread to service the io_uring requests/responses - Make it so every call to "normal" syscalls causes the calling thread to sleep until the result is available (that's 1 syscall) - When the executor thread gets a result, have it wake up the original thread (that's another syscall) So you're basically turning 1 syscall into 2 in order to emulate the legacy syscalls. io_uring only makes sense if you're already async. Emulating sync on top of async is nearly always a terrible idea.
- wtallis 1y ago
  You don't need to start spawning new threads to use io_uring as a backend for synchronous IO APIs. You just need to set up the rings once, then when the program does an fwrite or whatever, that gets implemented as sending a submission queue entry followed by a single io_uring_enter syscall that informs the kernel there's something in the submission queue, and using the arguments indicating that the calling process wants to block until there's something in the completion queue.
  - ninkendo 1y ago
    > using the arguments indicating the calling process wants to block Nice to know io_uring has facilities for backwards compatibility with blocking code here. But yeah, that's still a syscall, and given that the whole benefit of io_uring is in avoiding (or at least, coalescing) syscalls, I doubt having libc "just" use io_uring is going to give any tangible benefit.

rockorager 1y ago

Author of the project here! I have a little write up on this here: https://rockorager.dev/log/lsr-ls-but-with-io-uring https://rockorager.dev/log/lsr-ls-but-with-io-uring

jeffbee 1y ago
How much of the speedup over GNU ls is due to lacking localization features? Your results table is pretty much consistent with my local observations: in a dir with 13k files, `ls -al` needs 33ms. But 25% of that time is spent by libc in `strcoll`. Under `LC_ALL=C` it takes just 27ms, which is getting closer to the time of your program.
- rockorager 1y ago
  I didn't include `busybox` in my initial table, so it isn't on the blog post but the repo has the data...but I am 99% sure busybox does not have locale support, so I think GNU ls without locale support would probably be closer to busybox. Locales also bring in a lot more complicated sorting - so that could be a factor also.
tavianator 1y ago
My bfs project also uses io_uring: https://github.com/tavianator/bfs/blob/main/src/ioq.c https://github.com/tavianator/bfs/blob/main/src/ioq.c I'm curious how lsr compares to bfs -ls for example. bfs only uses io_uring when multiple threads are enabled, but maybe it's worth using it even for bfs -j1
- rockorager 1y ago
  Oh that's cool. `find` is another tool I thought could benefit from io_uring like `ls`. I think it's definitely worth enabling io_uring for single threaded applications for the batching benefit. The kernel will still spin up a thread pool to get the work done concurrently, but you don't have to manage that in your codebase.
  - 1y ago

jasonjmcghee 1y ago

I find it funny that there are icons for .mjs and .cjs file extensions but not .c, .h, .sh

swiftcoder 1y ago

Kind of fascinating that slashing syscalls by ~35x (versus the `ls -la` benchmark) is "only" worth a 2x speedup

bogwog 1y ago
I vaguely remember some benchmark I read a while back for some other io_uring project, and it suggested that io_uring syscalls are more expensive than whatever the other syscalls were that it was being used to replace. It's still a big improvement, even if not as big as you'd hope. I wish I could remember the post, but I've had that impression in the back of my mind ever since.
Galanwe 1y ago
These syscalls are mostly through VDSO, so not very costly
- wahern 1y ago
  The only VDSO-capable calls are clock_gettime, getcpu, getrandom, gettimeofday, and time. (Other architectures have some more, mostly related to signals and CPU cache flushing.)
Too 1y ago
Can someone try this in a virtual machine? I’d imagine syscalls there to be more expensive.

mnw21cam 1y ago

Love the idea and execution, don't love the misplaced apo'strophe's.

rockorager 1y ago
Oh no - where at?
- nbf_1995 1y ago
  Technically, the first, third, and fifth occurrence of "it's" should be "its". The dog chased its tail. I didn't notice when I read the article though. The original commenter is being pedantic.
- teo_zero 1y ago
  And since you're asking, beware of "let's" too: the third one is wrong!

pvtmert 1y ago

> I have no idea what lsd is doing. I haven’t read the source code, but from viewing it’s strace, it is calling clock_gettime around 5 times per file. Why? I don’t know. Maybe it’s doing internal timing of steps along the way? Maybe calculating "X minutes/hours/days/weeks ago" thing for each timestamp? (access, create, modify, ...). Could just be an old artifact of another library function...

namibj 1y ago
This shouldn't be an actual syscall these days; it should be handled by vDSO (`man 7 vDSO`). Maybe zig doesn't use that, though.

buybackoff 1y ago

A little offtop, but do you know a number in usecs that io_uring can save on enterprise grade servers, with 10G NICs, for socket latency overheads vs LD_PRELOAD when hardware supports that? Let's say it's Mellanox 4 or 5. My understanding is that each gives around 10us savings, maybe less. Based on some benchmarking, which was not focused on any of those explicitly but had some imprecise experiments. It also looks like they do not add up. Do you have a number based on real experience?

_ache_ 1y ago

Why doesn't it scale ? I'm really interested in n=100.000 and n=1.000.000. eza could catch up with lsr. Why is n=1.000 so slow with lsr ? Does io_uring add so much overhead ?

Lockal 1y ago

A relevant task in uutils https://github.com/uutils/coreutils/issues/4362 https://github.com/uutils/coreutils/issues/4362 describes how io_uring universally helps in almost any disk operation util.

michaelcampbell 1y ago

This is awesome for learning and a funzies app. Would be nice to see it moved into some mainstream distro, to be honest. That said, in over 35 years of using `ls`, never once have I bemoaned the speed of it.

flemhans 1y ago
Using ls in a dir with millions of files may stall a good while.
- michaelcampbell 1y ago
  Sure; I guess scaling personal things to Google scale just isn't something I've had to deal with as a pleb software dev/user.

snapplebobapple 1y ago

This is cool. I really miss the eza coloring of the file permissions/ownership/size/dates. I've installed it anyway to try when eza -l is eating it because of a large number of files in the directory. Side note: would something like this make ncdu faster? I run into large file counts = slow using that a lot more than the ls command.