5 ms·

Can anyone comment on how this interface compares to state-of-the-art async i/o in other *unix o/s? Is Linux playing catch up here or innovating?

crest 8y ago
Looks like a little bit of both. Linux has a lot to catch up because epoll is a mess. It is limited to file descriptors with a readiness semantic (sockets, pipes, ttys) meaning that it will never work with normal files because they are always ready while blocking whenever they feel like it. The second API fuck up is the fork/exec semantic. Oh and they tried to get away with edge triggered instead of level triggered. All in all epoll is a just a nasty case of NIH. The epoll developers just wrote code without understanding the prior art in their field (e.g. kqueue).
- AnanasAttack 8y ago
  What do you dislike about edge triggered mode?
- oralty 8y ago
  epoll (and even kqueue) is not aio and was never intended as such. While epoll had some early snafus, those have long been fixed (offering ET and LT) and it works well for its intended purpose (io readiness). This (io_uring) is more for a thing like Windows overlapped IO and "unix" (incl FreeBSD) aio. AIO on linux is yet a separate set of syscalls (io_*) and has since day one in the 2.5 days been pretty much a useless joke, very limited in scope (O_DIRECT only). kqueue is neither epoll nor aio but a common queueing interface for events, which includes io readiness and aio completion as separate types. Linux prefers to split these as different types of fds. FWIW, this wasn't due to obliviousness of the existing design - https://yarchive.net/comp/linux/event_queues.html https://yarchive.net/comp/linux/event_queues.html. Honestly your comment sounds very like a regurgitated old and somewhat outdated Bryan Cantrill rant.
  - gpderetta 8y ago
    The link doesn't explain much. The only argument that Linus has on kqueue is that is overengineered. Then it goes on a long multi message rant in how having multiple queues is insane. Luckily, in the end epoll ended up supporting multiple queues at least. I do believe that ET is good though.
asveikau 8y ago
I am not at up to date on this either, but as far as I know, the old status quo was this: For non-blocking sockets and the like you have what grew out of the readiness model from select and poll. Today on Linux that's epoll. This model never worked for files on disk. There is the POSIX aio API for that. On Linux, glibc implements that and it's always been based on synchronous I/O in thread pools because the kernel support isn't good. And I guess there's been Linux specific things like io_submit(2), which as the article states only worked with O_DIRECT and sometimes fell back to synchronous I/O. It says the new mechanism works with both files and sockets, but I imagine it's still reasonable to use epoll for sockets, and only use this new stuff for files on disk. One good thing about the readiness model for sockets is you don't need to allocate memory upfront every potential read the remote host might trigger.
- Klathmon 8y ago
  This explanation gave me that feeling of how utterly incredible it is at any software works as well as it does. The amount of work that goes into supporting, debugging, developing, improving, and designing the sheer number of ways that we do things, at literally hundreds of layers of abstractions all which are slightly wrong and each of which does things their own way with their own tradeoffs. I'm damn glad there are people a lot smarter and more dedicated than I am that keep this stuff running so that I can store a value in localstorage in a browser and have it punch through all of those layers in a fraction of a moment down to writing bits on one of like a dozen storage mediums on any number of OSs via multiple interfaces in multiple browsers.
- aarongolliver 8y ago
  Can you (or someone) explain _why_ the model translate to files on disk? Is it that there's a different set of operations that need to be supported? (you can't seek a socket I suppose)
  - asveikau 8y ago
    The model for aio is "here is a buffer, let me know when you've done the operation". Your request can sit in a queue somewhere or whatever for as long as it takes. The model for the various FD polling mechanisms is "wake me up when read(2) won't return EAGAIN when I call it later". That is very natural for sockets, because what read(2) is doing is draining a kernel side receive buffer, so the question is "wake me up when the buffer isn't empty". If the FD were a file on disk, knowing ahead of time that read(2) isn't going to return quickly is kind of a clumsy task.
snarfy 8y ago
Most of the problems with async i/o were spelled out very well in the recent "I/O is Faster Than CPU" [1] posted a few days ago. [1] - https://news.ycombinator.com/item?id=19818899 https://news.ycombinator.com/item?id=19818899 The ring buffer is definitely innovation.

crest 8y ago

Looks like a little bit of both. Linux has a lot to catch up because epoll is a mess. It is limited to file descriptors with a readiness semantic (sockets, pipes, ttys) meaning that it will never work with normal files because they are always ready while blocking whenever they feel like it. The second API fuck up is the fork/exec semantic. Oh and they tried to get away with edge triggered instead of level triggered. All in all epoll is a just a nasty case of NIH. The epoll developers just wrote code without understanding the prior art in their field (e.g. kqueue).

AnanasAttack 8y ago
What do you dislike about edge triggered mode?
oralty 8y ago
epoll (and even kqueue) is not aio and was never intended as such. While epoll had some early snafus, those have long been fixed (offering ET and LT) and it works well for its intended purpose (io readiness). This (io_uring) is more for a thing like Windows overlapped IO and "unix" (incl FreeBSD) aio. AIO on linux is yet a separate set of syscalls (io_*) and has since day one in the 2.5 days been pretty much a useless joke, very limited in scope (O_DIRECT only). kqueue is neither epoll nor aio but a common queueing interface for events, which includes io readiness and aio completion as separate types. Linux prefers to split these as different types of fds. FWIW, this wasn't due to obliviousness of the existing design - https://yarchive.net/comp/linux/event_queues.html https://yarchive.net/comp/linux/event_queues.html. Honestly your comment sounds very like a regurgitated old and somewhat outdated Bryan Cantrill rant.
- gpderetta 8y ago
  The link doesn't explain much. The only argument that Linus has on kqueue is that is overengineered. Then it goes on a long multi message rant in how having multiple queues is insane. Luckily, in the end epoll ended up supporting multiple queues at least. I do believe that ET is good though.

asveikau 8y ago

I am not at up to date on this either, but as far as I know, the old status quo was this: For non-blocking sockets and the like you have what grew out of the readiness model from select and poll. Today on Linux that's epoll. This model never worked for files on disk. There is the POSIX aio API for that. On Linux, glibc implements that and it's always been based on synchronous I/O in thread pools because the kernel support isn't good. And I guess there's been Linux specific things like io_submit(2), which as the article states only worked with O_DIRECT and sometimes fell back to synchronous I/O. It says the new mechanism works with both files and sockets, but I imagine it's still reasonable to use epoll for sockets, and only use this new stuff for files on disk. One good thing about the readiness model for sockets is you don't need to allocate memory upfront every potential read the remote host might trigger.

Klathmon 8y ago
This explanation gave me that feeling of how utterly incredible it is at any software works as well as it does. The amount of work that goes into supporting, debugging, developing, improving, and designing the sheer number of ways that we do things, at literally hundreds of layers of abstractions all which are slightly wrong and each of which does things their own way with their own tradeoffs. I'm damn glad there are people a lot smarter and more dedicated than I am that keep this stuff running so that I can store a value in localstorage in a browser and have it punch through all of those layers in a fraction of a moment down to writing bits on one of like a dozen storage mediums on any number of OSs via multiple interfaces in multiple browsers.
aarongolliver 8y ago
Can you (or someone) explain _why_ the model translate to files on disk? Is it that there's a different set of operations that need to be supported? (you can't seek a socket I suppose)
- asveikau 8y ago
  The model for aio is "here is a buffer, let me know when you've done the operation". Your request can sit in a queue somewhere or whatever for as long as it takes. The model for the various FD polling mechanisms is "wake me up when read(2) won't return EAGAIN when I call it later". That is very natural for sockets, because what read(2) is doing is draining a kernel side receive buffer, so the question is "wake me up when the buffer isn't empty". If the FD were a file on disk, knowing ahead of time that read(2) isn't going to return quickly is kind of a clumsy task.

snarfy 8y ago

Most of the problems with async i/o were spelled out very well in the recent "I/O is Faster Than CPU" [1] posted a few days ago. [1] - https://news.ycombinator.com/item?id=19818899 https://news.ycombinator.com/item?id=19818899 The ring buffer is definitely innovation.