6 ms·

How would the performance of io_uring would merit its complexity for a terminal file manager?

ithkuil 3y ago
Wouldn't io_uring complexity be hidden behind the async io abstraction? Would the file manager even have to bother what underlying mechanism provides the async IO interface with the kernel? (epoll, kqueue, io_uring, etc) Are there know ways that io_uring would leak through such an abstraction?
- pxeger1 3y ago
  I was assuming the file manager would have to include the code for the async abstraction. But of course it could be just part of the async runtime / standard library. Still, a file manager doesn't seem like it would really need the ability to be so massively concurrent.
  - remram 3y ago
    I'm also lost. Does it just mean that the UI stays responsive while a file copy happens in the background? It says "CPU tasks are spread across multiple threads", but what CPU tasks does a file manager perform?
    - samus 3y ago
      Almost every UI framework out there is single-threaded and has facilities to help offloading such tasks into the background. Usually, to a worker thread pool. It's a bit fiddly, especially in a non-managed language that doesn't protect from dumb mistakes, but it can be easily done. io_uring is for use cases where the context switch overhead of system calls starts to matter. While async I/O can be justified for U/I programming, it's safe to say that a file manager won't ever benefit from io_uring.
    - sxyazi 3y ago
      There is a Reddit post discussing this point: https://www.reddit.com/r/rust/comments/16fxr58/comment/k04u7z8/?context=3 https://www.reddit.com/r/rust/comments/16fxr58/comment/k04u7... > There are many scenarios that require CPU, such as image downscaling, image encoding, code highlighting, sorting large directories, etc. If you want to delve further into Yazi's inner workings, you can see https://github.com/sxyazi/yazi/issues/143 https://github.com/sxyazi/yazi/issues/143
      - remram 3y ago
        What advantage does thread pool + io_uring have over just thread pool? Unless you have millions of CPU cores and therefore many downscaling/encoding/highlighting tasks starting/finishing every second, I still don't see what io_uring brings to the table. Running tasks in the background does NOT require io_uring.
        sxyazi 3y ago
        Hey, Yazi currently is NOT using io_uring, and the link provided above explains it thoroughly, please take a careful look.

Wouldn't io_uring complexity be hidden behind the async io abstraction? Would the file manager even have to bother what underlying mechanism provides the async IO interface with the kernel? (epoll, kqueue, io_uring, etc) Are there know ways that io_uring would leak through such an abstraction?

pxeger1 3y ago
I was assuming the file manager would have to include the code for the async abstraction. But of course it could be just part of the async runtime / standard library. Still, a file manager doesn't seem like it would really need the ability to be so massively concurrent.
- remram 3y ago
  I'm also lost. Does it just mean that the UI stays responsive while a file copy happens in the background? It says "CPU tasks are spread across multiple threads", but what CPU tasks does a file manager perform?
  - samus 3y ago
    Almost every UI framework out there is single-threaded and has facilities to help offloading such tasks into the background. Usually, to a worker thread pool. It's a bit fiddly, especially in a non-managed language that doesn't protect from dumb mistakes, but it can be easily done. io_uring is for use cases where the context switch overhead of system calls starts to matter. While async I/O can be justified for U/I programming, it's safe to say that a file manager won't ever benefit from io_uring.
  - sxyazi 3y ago
    There is a Reddit post discussing this point: https://www.reddit.com/r/rust/comments/16fxr58/comment/k04u7z8/?context=3 https://www.reddit.com/r/rust/comments/16fxr58/comment/k04u7... > There are many scenarios that require CPU, such as image downscaling, image encoding, code highlighting, sorting large directories, etc. If you want to delve further into Yazi's inner workings, you can see https://github.com/sxyazi/yazi/issues/143 https://github.com/sxyazi/yazi/issues/143
    - remram 3y ago
      What advantage does thread pool + io_uring have over just thread pool? Unless you have millions of CPU cores and therefore many downscaling/encoding/highlighting tasks starting/finishing every second, I still don't see what io_uring brings to the table. Running tasks in the background does NOT require io_uring.
      - sxyazi 3y ago
        Hey, Yazi currently is NOT using io_uring, and the link provided above explains it thoroughly, please take a careful look.