4 ms·

This feels very very similar to IO completion ports / iocp on Windows. More modern versions of Windows even has registered buffers for completion which can be e

by adzm 6y ago

This feels very very similar to IO completion ports / iocp on Windows. More modern versions of Windows even has registered buffers for completion which can be even more performant in certain scenarios. I'm looking forward to trying this out on Linux. I'm curious to see how this might work its way into libuv and c++ ASIO libraries, too.

leetrout 6y ago
libuv is tracking in https://github.com/libuv/libuv/issues/1947 https://github.com/libuv/libuv/issues/1947
- cycloptic 6y ago
  Tracking that issue was a motivator for me to begin adding support to glib: https://gitlab.gnome.org/GNOME/glib/-/issues/2084 https://gitlab.gnome.org/GNOME/glib/-/issues/2084
ithkuil 6y ago
io_uring allows the kernel and the user program to communicate purely via shared memory without having to perform a system call, i.e. a context switch to the kernel. Do windows completion ports also work that way or do they involve a system call to be performed in order to consume completion events?
- ta8645 6y ago
  You do still need a single system call (io_uring_submit) to submit each batch of entries in the submission queue. Edit: actually no it's not required in all cases. Thanks for the correction.
  - WJW 6y ago
    You can ask the kernel to poll the submission queue and skip io_uring_submit too. (Though you need elevated privileges to do this IIRC)
  - ithkuil 6y ago
    I only read about io_uring without yet having a chance of actually using it so take this with a grain of salt: I read that io_uring has two modes, one where you signal via a system call and another that uses memory mapped polling. https://unixism.net/loti/tutorial/sq_poll.html https://unixism.net/loti/tutorial/sq_poll.html states: > Reducing the number of system calls is a major aim for io_uring. To this end, io_uring lets you submit I/O requests without you having to make a single system call. This is done via a special submission queue polling feature that io_uring supports.
  - yxhuvud 6y ago
    Submit is not a syscall. io_uring_enter is the only syscall that is used while running a ring. That one may submit, wait or both at the same time. Strictly speaking it isn't necessary but to avoid it you require elevated privileges.
    - frevib 6y ago
      From Linux 5.10 you only need CAP_SYS_NICE to perform SQPOLL: https://git.kernel.dk/cgit/linux-block/commit/?h=io_uring-files_struct&id=03a54c18518d196693217aa9f683369eacb01daf https://git.kernel.dk/cgit/linux-block/commit/?h=io_uring-fi...
      - yxhuvud 6y ago
        Yes, hence "elevated" and not "root". It is still higher than default, right?
  - junon 6y ago
    Not with SQPOLL. You can eliminate all syscalls with SQPOLL.
- Matthias247 6y ago
  Windows registered IO (RIO) does imho the same (https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/hh997032(v=ws.11) https://docs.microsoft.com/en-us/previous-versions/windows/i...). When enqueuing reads/writes with RIO there at least exist flags to specify that the kernel should not immediately be woken up, and thereby to batch syscalls as with io_uring.
cyphar 6y ago
You're quite right -- it's basically the same idea as IOCP on Windows, kqueue on FreeBSD, and Event Ports on Solaris.
- Matthias247 6y ago
  Isn't kqueue for sockets still readiness based? I know most runtimes (like libuv) just use it in the same fashion as epoll, and await readability/writeability through the queue. Not sure if it also has completion based options.
  - cyphar 6y ago
    You're quite right, I muddled things up (and now the edit window has elapsed). epoll is the Linux equivalent of kqueue, IOCP, and Event Ports (all readiness based). Not sure how I screwed that one up...
    - Matthias247 6y ago
      IOCP is actually submission+completion based and thereby closer to io_uring than to epoll. The main difference between IOCP and io_uring at this point seems to be the use of a ringbuffer based submit interface instead of a syscall based one. But that is more of a performance optimization than a huge difference in the programming model.
ncmncm 6y ago
It is already integrated with asio. Third-party, of course, because that's the whole point: io_uring does not need to know anything about asio, nor does asio need to know anything about io_uring, to get optimal performance. It's all on github, with accompanying CppCon talk. Asio, by the way, will be C++23's network layer.

by adzm 6y ago

leetrout 6y ago

libuv is tracking in https://github.com/libuv/libuv/issues/1947 https://github.com/libuv/libuv/issues/1947

cycloptic 6y ago
Tracking that issue was a motivator for me to begin adding support to glib: https://gitlab.gnome.org/GNOME/glib/-/issues/2084 https://gitlab.gnome.org/GNOME/glib/-/issues/2084

ithkuil 6y ago

io_uring allows the kernel and the user program to communicate purely via shared memory without having to perform a system call, i.e. a context switch to the kernel. Do windows completion ports also work that way or do they involve a system call to be performed in order to consume completion events?

ta8645 6y ago
You do still need a single system call (io_uring_submit) to submit each batch of entries in the submission queue. Edit: actually no it's not required in all cases. Thanks for the correction.
- WJW 6y ago
  You can ask the kernel to poll the submission queue and skip io_uring_submit too. (Though you need elevated privileges to do this IIRC)
- ithkuil 6y ago
  I only read about io_uring without yet having a chance of actually using it so take this with a grain of salt: I read that io_uring has two modes, one where you signal via a system call and another that uses memory mapped polling. https://unixism.net/loti/tutorial/sq_poll.html https://unixism.net/loti/tutorial/sq_poll.html states: > Reducing the number of system calls is a major aim for io_uring. To this end, io_uring lets you submit I/O requests without you having to make a single system call. This is done via a special submission queue polling feature that io_uring supports.
- yxhuvud 6y ago
  Submit is not a syscall. io_uring_enter is the only syscall that is used while running a ring. That one may submit, wait or both at the same time. Strictly speaking it isn't necessary but to avoid it you require elevated privileges.
  - frevib 6y ago
    From Linux 5.10 you only need CAP_SYS_NICE to perform SQPOLL: https://git.kernel.dk/cgit/linux-block/commit/?h=io_uring-files_struct&id=03a54c18518d196693217aa9f683369eacb01daf https://git.kernel.dk/cgit/linux-block/commit/?h=io_uring-fi...
    - yxhuvud 6y ago
      Yes, hence "elevated" and not "root". It is still higher than default, right?
- junon 6y ago
  Not with SQPOLL. You can eliminate all syscalls with SQPOLL.
Matthias247 6y ago
Windows registered IO (RIO) does imho the same (https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/hh997032(v=ws.11) https://docs.microsoft.com/en-us/previous-versions/windows/i...). When enqueuing reads/writes with RIO there at least exist flags to specify that the kernel should not immediately be woken up, and thereby to batch syscalls as with io_uring.

cyphar 6y ago

You're quite right -- it's basically the same idea as IOCP on Windows, kqueue on FreeBSD, and Event Ports on Solaris.

Matthias247 6y ago
Isn't kqueue for sockets still readiness based? I know most runtimes (like libuv) just use it in the same fashion as epoll, and await readability/writeability through the queue. Not sure if it also has completion based options.
- cyphar 6y ago
  You're quite right, I muddled things up (and now the edit window has elapsed). epoll is the Linux equivalent of kqueue, IOCP, and Event Ports (all readiness based). Not sure how I screwed that one up...
  - Matthias247 6y ago
    IOCP is actually submission+completion based and thereby closer to io_uring than to epoll. The main difference between IOCP and io_uring at this point seems to be the use of a ringbuffer based submit interface instead of a syscall based one. But that is more of a performance optimization than a huge difference in the programming model.

ncmncm 6y ago

It is already integrated with asio. Third-party, of course, because that's the whole point: io_uring does not need to know anything about asio, nor does asio need to know anything about io_uring, to get optimal performance. It's all on github, with accompanying CppCon talk. Asio, by the way, will be C++23's network layer.