4 ms·
Asynchronous I/O completion notification was a huge innovation, allowing computation to proceed concurrently with I/O and thus dramatically increasing the throu
by kragen 2y ago
Asynchronous I/O completion notification was a huge innovation, allowing computation to proceed concurrently with I/O and thus dramatically increasing the throughput of the computer. Unfortunately it was also a huge source of bugs. The following innovation that made it usable, by making it less bug-prone, was called a "multitasking operating system". The so-called "OS" allowed you to write simple sequential code, but used the computer efficiently by switching back and forth between multiple tasks as their respective I/Os completed. We're talking about the introduction of the Univac 1103A in 01953, 72 years ago, and the following 20 years of innovations, including things like Dijkstra's THE operating system.
That is, asynchronous I/O is 20 years older than the Unix system call interface this article speculates it should replace.
Of course, context switching between different tasks is not free, and event loops have frequently been able to provide higher efficiency. The equilibrium has rocked back and forth as I/O has gotten faster and slower relative to task context switching. CICS, select(), poll(), Oberon, the Macintosh system, Win16, the JavaScript event loop, Tcl/Tk, Win32 IOCP, Symbian active objects, kqueue, epoll, and io_uring are some of the results.
But don't try to sell asynchronous I/O as a "game-changing" paradigm shift. It's a different programming model that's harder to program but can provide higher performance, just like it has been for 70 years.
If you're shopping for a paradigm shift that can improve this tradeoff, there are several candidates. Erlang-style lightweight processes, software transactional memory, and JS-style promises (originally from E, which was inspired by KeyKOS, which had event loops but no promises or asynchronous I/O) come to mind.
The hardware development that might be actually new has arguably already failed in the market: Intel/Micron's "Optane" memory and Flash-based NVDIMMs. Flash has big disk energy, but is fast enough that copying it to RAM one word at a time like disk will probably bottleneck your performance by an order of magnitude. io_uring doesn't fix this. We need interfaces designed for zero-copy access to bulk persistent data. Maybe something like Multics or mmap(). LMDB and FlatBuffers suggest that the potential for improved performance is significant. Could such high-bandwidth, low-cost memory keep up with LLM inference, allowing you to do inference on a 256-gigabyte model with 256 gigabytes of Flash but a much smaller amount of RAM?
- baruch 2y agoAsynchronous IO with user space threads works wonders to get both the performance of async IO and the convenience of sequential programming.
- mpweiher 2y ago1. The article wasn't talking about "asynchronous I/O". It was talking about message-passing (actor model/message-oriented OO) as the interface. 2. Message passing isn't new, it is at least 50 years old even if you don't go further back than early Smalltalk and the Actor model. 3. The article wasn't "selling" anything, and certainly nothing new see (2). It was noting convergence from several directions on an old and somewhat misunderstood paradigm.
- kragen 2y agoIt mentions the names of message-passing and the well-known actors model, but the ideas it engages with are the ideas of io_uring, which is a system for asynchronous I/O using a message queue. The two designs are not unrelated, and you could write an interesting post exploring the relationships between them, but unfortunately the author did not do that, because his engagement with the ideas of the actor model and message-passing was limited to quoting one of Hewitt and Baker's famous early attempts to define actors. Instead, he jumps into salesman hypester mode with "The game has changed," which, like, gag. A lot of games are having their rules rewritten right now (tank/drone warfare, the energy market, freedom of expression, international finance, and artificial intelligence come to mind) but asynchronous I/O is not one of them. (Except, maybe, in the non-io_uring-related way I suggested—the advent of much-higher-bandwidth access to large-capacity storage devices than I/O buses can handle—to which message-passing is even less applicable.) It's perfectly fair to describe actors as "somewhat misunderstood" because the ways Hewitt himself understood it over the 50 years he developed the idea frequently contradict one another. At the end of his life he spent several years writing https://arxiv.org/abs/0904.3036v12 https://arxiv.org/abs/0904.3036v12, which describes his conceptualization of it at that time, which was very different from the early versions, though I think he would deny that. The versions on the arXiv only go back to 02009, but I am pretty sure the draft paper he showed me when I met him a few years before that was an earlier draft of the same thing.