5 ms·

> non-blocking semantics and poll/select suck. I'd argue that it's not that the semantics suck, it's that there are too many of them. We have O_NONBLOCK, multi

> non-blocking semantics and poll/select suck. I'd argue that it's not that the semantics suck, it's that there are too many of them. We have O_NONBLOCK, multiple multiplexers (for sane reasons--I don't begrudge 1980s folks for not thinking about fd set size and copy overhead for select(2) either), and others. What's worse, they don't all work with all FDs--not only are regular files not nonblock-able in the same way that sockets are, but all sorts of other FD-exposed capabilities (signalfds, timerfds, memfds, pidfds) do or don't support nonblocking semantics and multiplexing in all sorts of weird ways. If those old system designers had stuck with keeping the async IO syscall space small (e.g. "you only get select/poll" or "you only get read(fds) and read_noblock(fds, timeout)") and consistent (by drawing a hard line at "if you expose something as an FD, it must support all APIs that generically handle FDs"), we would have ended up in a better place. Sure, that would have slowed down some implementations (e.g. vfs drivers), but would also have massively sped up development against a lot of these APIs. Ah, well, hindsight is 20/20 I guess.

adrian_b 2mo ago
Allready in 1983, 4.2BSD had 3 different attempts at doing multiplexed/asynchronous I/O: Non-blocking I/O: "O_NONBLOCK" with "EWOULDBLOCK" (or "EAGAIN") Signal-driven I/O: "O_ASYNC" with "SIGIO" Synchronous I/O multiplexing: "select()" All 3 had various defects, especially with a large number of concurrent I/O actions. In UNIX-derived operating systems, after 1983 there have been many other attempts to implement something better than these 3 (starting with System V "poll" and with POSIX AIO, and then with various incompatible approaches in Solaris, FreeBSD and Linux), but none were good enough and most were seriously inferior to methods of doing asynchronous I/O that existed in some IBM and DEC operating systems decades earlier. In my opinion, only io_uring has finally solved the problem of multiplex asynchronous I/O in Linux, and in a manner much better than in all older operating systems. I consider all the many older alternatives that exist for liburing as obsolete.
- sscaryterry 2mo ago
  This. I'd break-out in a cold sweat if I had to work on this again.
- celiacFun 2mo ago
  Windows has a better async I/O implementation than Linux and has for decades
  - jchw 2mo ago
    No, although this was true until around 2019 when Linux introduced io_uring and finally had a genuine innovation. io_uring wound up giving us a unified asynchronous I/O layer that handles just tons of things. (And it is genuinely a step up from the overlapped I/O APIs of Windows, as it avoids eating syscall transition overhead in many cases.) Of course, Microsoft then went on to introduce IORing in 2022, continuing their legacy of never being afraid to adopt good ideas. But still - it would be wrong to suggest Linux is currently behind on async I/O. It hasn't been for years.
inigyou 2mo ago
What would all the other FDs be instead? Like an eventfd or epoll handle, it obviously couldn't be an FD so what would it actually be?
- zbentley 2mo ago
  Sorry, what I meant was that FDs that aren’t sockets should be uniformly handled by the async IO syscalls rather than having to parse man pages or trip over EINVALs to figure out the answers to questions like “can I select(2) on a pidfd? Can I O_NOBLOCK a regular on-disk file?”. For a concrete example, I should be able to select/poll/epoll any file descriptor. The multiplexers can short-circuit return for things where readiness is inherent (e.g. block device files/vfs, I’m not asking for the moon a la waiting for NFS shares to report ready or something). I should be able to issue non blocking reads on special file descriptors (timerfds, signalfds, eventfds, and so on). This is analogous to my other unachievable fantasy: everything that exposes a filesystem interface must work with inotify/kqueue/whatever. No exceptions for procfs/NFS/etc. At some point this fantasizing just becomes me griping about what made it to market/the worse-is-better philosophy generally, though. And yet it moves, I guess!
  - inigyou 2mo ago
    Or you throw it all out and do Windows-style overlapped I/O instead.

adrian_b 2mo ago

Allready in 1983, 4.2BSD had 3 different attempts at doing multiplexed/asynchronous I/O: Non-blocking I/O: "O_NONBLOCK" with "EWOULDBLOCK" (or "EAGAIN") Signal-driven I/O: "O_ASYNC" with "SIGIO" Synchronous I/O multiplexing: "select()" All 3 had various defects, especially with a large number of concurrent I/O actions. In UNIX-derived operating systems, after 1983 there have been many other attempts to implement something better than these 3 (starting with System V "poll" and with POSIX AIO, and then with various incompatible approaches in Solaris, FreeBSD and Linux), but none were good enough and most were seriously inferior to methods of doing asynchronous I/O that existed in some IBM and DEC operating systems decades earlier. In my opinion, only io_uring has finally solved the problem of multiplex asynchronous I/O in Linux, and in a manner much better than in all older operating systems. I consider all the many older alternatives that exist for liburing as obsolete.

sscaryterry 2mo ago
This. I'd break-out in a cold sweat if I had to work on this again.
celiacFun 2mo ago
Windows has a better async I/O implementation than Linux and has for decades
- jchw 2mo ago
  No, although this was true until around 2019 when Linux introduced io_uring and finally had a genuine innovation. io_uring wound up giving us a unified asynchronous I/O layer that handles just tons of things. (And it is genuinely a step up from the overlapped I/O APIs of Windows, as it avoids eating syscall transition overhead in many cases.) Of course, Microsoft then went on to introduce IORing in 2022, continuing their legacy of never being afraid to adopt good ideas. But still - it would be wrong to suggest Linux is currently behind on async I/O. It hasn't been for years.

inigyou 2mo ago

What would all the other FDs be instead? Like an eventfd or epoll handle, it obviously couldn't be an FD so what would it actually be?

zbentley 2mo ago
Sorry, what I meant was that FDs that aren’t sockets should be uniformly handled by the async IO syscalls rather than having to parse man pages or trip over EINVALs to figure out the answers to questions like “can I select(2) on a pidfd? Can I O_NOBLOCK a regular on-disk file?”. For a concrete example, I should be able to select/poll/epoll any file descriptor. The multiplexers can short-circuit return for things where readiness is inherent (e.g. block device files/vfs, I’m not asking for the moon a la waiting for NFS shares to report ready or something). I should be able to issue non blocking reads on special file descriptors (timerfds, signalfds, eventfds, and so on). This is analogous to my other unachievable fantasy: everything that exposes a filesystem interface must work with inotify/kqueue/whatever. No exceptions for procfs/NFS/etc. At some point this fantasizing just becomes me griping about what made it to market/the worse-is-better philosophy generally, though. And yet it moves, I guess!
- inigyou 2mo ago
  Or you throw it all out and do Windows-style overlapped I/O instead.