5 ms·

io_uring will help a lot here.

by khuey 6y ago

klodolph 6y ago
Well, it can, but not always. Remember that if you’re waiting for an event to arrive, that generally involves a syscall, the thread being put to sleep, and then the thread being woken up. Any time you’re doing that, think, “Could I just replace this polling system with a call to read()?” What io_uring does do is provide a way to poll without needing to wait, but if you haven’t received new events when you poll, you’re not on the fast path any more. Whether you are often on the fast path for io_uring will depend on the particulars of your application and its I/O patterns.
- quotemstr 6y ago
  io_uring is specifically designed so that zero system calls are necessary while the system is busy. Userspace and the kernel both update ring buffers, and ring buffers can be checked and drained without entering the kernel at all.
  - klodolph 6y ago
    Yes, that’s what “if you haven’t received new events when you poll” means.
- scottlamb 6y ago
  > What io_uring does do is provide a way to poll without needing to wait, but if you haven’t received new events when you poll, you’re not on the fast path any more. Isn't "not on the fast path any more" a bit absolutist? io_uring's "slow" path is roughly one syscall per iteration, right? That's still many fewer syscalls than one syscall per IO operation (or more if any return EAGAIN/EWOULDBLOCK) as you'd be doing without it. I'm not sure I really care about eliminating that last syscall per iteration; it seems minor in comparison.
  - klodolph 6y ago
    io_uring’s slow path is making one blocking syscall every time you would ordinarily make a blocking syscall. I am a bit baffled how this could possibly be considered an “absolutist” viewpoint—I am just saying that there exist scenarios where io_uring is not helpful. This should be uncontroversial.
    - scottlamb 6y ago
      You mean when there's only one thing to do per iteration? I'd describe that as when mostly idle. As the system gets more loaded, the one syscall per iteration matters less and less.
    - jorangreef 6y ago
      > io_uring’s slow path is making one blocking syscall every time you would ordinarily make a blocking syscall. That's not correct, io_uring was "absolutely" designed, at least in the technical sense, for zero syscalls in the slow path (if you want to): IORING_SETUP_SQPOLL When this flag is specified, a kernel thread is created to perform submission queue polling. An io_uring instance configured in this way enables an application to issue I/O without ever context switching into the kernel. By using the submission queue to fill in new submission queue entries and watching for completions on the completion queue, the application can submit and reap I/Os without doing a single system call. From the man page: https://manpages.debian.org/unstable/liburing-dev/io_uring_setup.2.en.html https://manpages.debian.org/unstable/liburing-dev/io_uring_s... This mode required privileges in early kernel versions but that's already changed. Things are moving fast.
  - volta83 6y ago
    > io_uring's "slow" path is roughly one syscall per iteration, right? Right, so if a blocking API makes 1 syscall, io_uring would make N syscals for N iterations.
    - jorangreef 6y ago
      Where 1 iteration and 1 syscall to io_uring_enter() is submitting 100s of I/O operations per io_uring_enter() syscall (and you can even run the ring buffers with the kernel set to poll so you can do 0 syscalls if that's not already enough). That's pretty huge amortization. Rough benchmarks we've done are showing double throughput for io_uring for 4096 byte AF sector write/fsync/read combos: https://github.com/coilhq/tigerbeetle/tree/master/demos/io_uring https://github.com/coilhq/tigerbeetle/tree/master/demos/io_u...
the8472 6y ago
io_uring doesn't mean async though. You can also use it for blocking batch execution of syscalls. E.g. when you need to stat hundreds of files or wait for several child processes at once. So with some batch-oriented convenience wrappers it can help threaded code too.

io_uring will help a lot here.

by khuey 6y ago

io_uring will help a lot here.

Well, it can, but not always. Remember that if you’re waiting for an event to arrive, that generally involves a syscall, the thread being put to sleep, and then the thread being woken up. Any time you’re doing that, think, “Could I just replace this polling system with a call to read()?” What io_uring does do is provide a way to poll without needing to wait, but if you haven’t received new events when you poll, you’re not on the fast path any more. Whether you are often on the fast path for io_uring will depend on the particulars of your application and its I/O patterns.

quotemstr 6y ago
io_uring is specifically designed so that zero system calls are necessary while the system is busy. Userspace and the kernel both update ring buffers, and ring buffers can be checked and drained without entering the kernel at all.
- klodolph 6y ago
  Yes, that’s what “if you haven’t received new events when you poll” means.
scottlamb 6y ago
> What io_uring does do is provide a way to poll without needing to wait, but if you haven’t received new events when you poll, you’re not on the fast path any more. Isn't "not on the fast path any more" a bit absolutist? io_uring's "slow" path is roughly one syscall per iteration, right? That's still many fewer syscalls than one syscall per IO operation (or more if any return EAGAIN/EWOULDBLOCK) as you'd be doing without it. I'm not sure I really care about eliminating that last syscall per iteration; it seems minor in comparison.
- klodolph 6y ago
  io_uring’s slow path is making one blocking syscall every time you would ordinarily make a blocking syscall. I am a bit baffled how this could possibly be considered an “absolutist” viewpoint—I am just saying that there exist scenarios where io_uring is not helpful. This should be uncontroversial.
  - scottlamb 6y ago
    You mean when there's only one thing to do per iteration? I'd describe that as when mostly idle. As the system gets more loaded, the one syscall per iteration matters less and less.
  - jorangreef 6y ago
    > io_uring’s slow path is making one blocking syscall every time you would ordinarily make a blocking syscall. That's not correct, io_uring was "absolutely" designed, at least in the technical sense, for zero syscalls in the slow path (if you want to): IORING_SETUP_SQPOLL When this flag is specified, a kernel thread is created to perform submission queue polling. An io_uring instance configured in this way enables an application to issue I/O without ever context switching into the kernel. By using the submission queue to fill in new submission queue entries and watching for completions on the completion queue, the application can submit and reap I/Os without doing a single system call. From the man page: https://manpages.debian.org/unstable/liburing-dev/io_uring_setup.2.en.html https://manpages.debian.org/unstable/liburing-dev/io_uring_s... This mode required privileges in early kernel versions but that's already changed. Things are moving fast.
- volta83 6y ago
  > io_uring's "slow" path is roughly one syscall per iteration, right? Right, so if a blocking API makes 1 syscall, io_uring would make N syscals for N iterations.
  - jorangreef 6y ago
    Where 1 iteration and 1 syscall to io_uring_enter() is submitting 100s of I/O operations per io_uring_enter() syscall (and you can even run the ring buffers with the kernel set to poll so you can do 0 syscalls if that's not already enough). That's pretty huge amortization. Rough benchmarks we've done are showing double throughput for io_uring for 4096 byte AF sector write/fsync/read combos: https://github.com/coilhq/tigerbeetle/tree/master/demos/io_uring https://github.com/coilhq/tigerbeetle/tree/master/demos/io_u...

the8472 6y ago

io_uring doesn't mean async though. You can also use it for blocking batch execution of syscalls. E.g. when you need to stat hundreds of files or wait for several child processes at once. So with some batch-oriented convenience wrappers it can help threaded code too.