4 ms·

I think a lot of folks think of Network IO when they say Asynchronous IO. That's only half the story, unless you're just building proxies and caches you have to

by mtanski 11y ago

I think a lot of folks think of Network IO when they say Asynchronous IO. That's only half the story, unless you're just building proxies and caches you have to deal with Disk IO at some point in time. And, async disk IO is horrible in every OS / language.

saaadhu 11y ago
Care to explain why async disk IO is horrible? AFAIK, the kernel APIs only deal with abstract handles and descriptors - they aren't written specifically for disk IO or network IO. And C#'s async/await and BeginXXX also work fine with all kinds of IO.
- kentonv 11y ago
  On Unix, the APIs for async disk I/O are completely different from the APIs for async network I/O. This is because, on Unix, the APIs for network I/O are "readiness"-based (they tell you when data is available in the read buffer or space is available in the write buffer), but a disk file handle is always ready (except at EOF). So, for disk I/O you need a "completion" model, where you queue an operation and then get a callback when it is done. This is a very different kind of API, and as a result it does not fit well with the network I/O APIs, but of course if you are doing async you probably need to do both disk and network at the same time, so you need the APIs to play nice with each other (you need a single event loop that waits for both kinds of events). (On Windows, as I understand it, you can do completion-based I/O on all handle types.) And then, even if you manage to use async disk I/O, you can only really use it for reads and writes. Filesystem calls that manipulate the directory tree usually don't have async versions. At some point you have to give up and do things in threads.
stuaxo 11y ago
The new async keyword in python looks a lot nicer than what we had before.
justincormack 11y ago
Well, apparently not on Windows. But this is a kernel interface issue not a language issue, and clearly needs fixing now we have SSDs where having thousands of outstanding requests is useful, if not required for performance, unless the hardware APIs are going to change (maybe if they get memory interfaces this does change).
- mtanski 11y ago
  The async story on windows is better, but not great. In many cases the async calls on windows silently block due to a number of special cases (that are not so special).
  - trentnelson 11y ago
    (Presuming you're referring to this: https://support.microsoft.com/en-us/kb/156932 https://support.microsoft.com/en-us/kb/156932) Those are all entirely reasonable things to block for if you understand the mechanics behind why those calls may block. And none of it should be fast-path stuff; it's easy to architect a solution where you avoid those things except in corner cases. The other thing you're missing is that when you architect around Windows completion ports and threadpool I/O facilities, it doesn't matter if one thread blocks every so often. Windows will detect this and schedule another one to run, such that there is one active thread scheduled on every core. I/O completion ports facilitate thread-agnostic I/O completion; the thread that blocked because it was extending an encrypted file won't impede the latency of other clients because there is no thread-specific association. I exploit all of this with PyParallel. http://pyparallel.org http://pyparallel.org For more details on asynchronous I/O on Windows and why it's fundamentally better than the UNIX I/O model in every possible way: https://speakerdeck.com/trent/pyparallel-how-we-removed-the-gil-and-exploited-all-cores?slide=47 https://speakerdeck.com/trent/pyparallel-how-we-removed-the-...
- trentnelson 11y ago
  > Well, apparently not on Windows. Agreed! http://pyparallel.org http://pyparallel.org > But this is a kernel interface issue not a language issue, Absolutely agree! The thing that surprises me most about Rust and Go in particular is that they're trying to solve an OS problem with new language abstractions. You don't need a new language -- what you really want is thread-agnostic I/O primitives, thread-agnostic "threadpool" primitives, and completion-oriented APIs that provide both synchronous and asynchronous primitives. Basically, you want what Windows has.

by mtanski 11y ago

saaadhu 11y ago

Care to explain why async disk IO is horrible? AFAIK, the kernel APIs only deal with abstract handles and descriptors - they aren't written specifically for disk IO or network IO. And C#'s async/await and BeginXXX also work fine with all kinds of IO.

kentonv 11y ago
On Unix, the APIs for async disk I/O are completely different from the APIs for async network I/O. This is because, on Unix, the APIs for network I/O are "readiness"-based (they tell you when data is available in the read buffer or space is available in the write buffer), but a disk file handle is always ready (except at EOF). So, for disk I/O you need a "completion" model, where you queue an operation and then get a callback when it is done. This is a very different kind of API, and as a result it does not fit well with the network I/O APIs, but of course if you are doing async you probably need to do both disk and network at the same time, so you need the APIs to play nice with each other (you need a single event loop that waits for both kinds of events). (On Windows, as I understand it, you can do completion-based I/O on all handle types.) And then, even if you manage to use async disk I/O, you can only really use it for reads and writes. Filesystem calls that manipulate the directory tree usually don't have async versions. At some point you have to give up and do things in threads.

stuaxo 11y ago

The new async keyword in python looks a lot nicer than what we had before.

justincormack 11y ago

Well, apparently not on Windows. But this is a kernel interface issue not a language issue, and clearly needs fixing now we have SSDs where having thousands of outstanding requests is useful, if not required for performance, unless the hardware APIs are going to change (maybe if they get memory interfaces this does change).

mtanski 11y ago
The async story on windows is better, but not great. In many cases the async calls on windows silently block due to a number of special cases (that are not so special).
- trentnelson 11y ago
  (Presuming you're referring to this: https://support.microsoft.com/en-us/kb/156932 https://support.microsoft.com/en-us/kb/156932) Those are all entirely reasonable things to block for if you understand the mechanics behind why those calls may block. And none of it should be fast-path stuff; it's easy to architect a solution where you avoid those things except in corner cases. The other thing you're missing is that when you architect around Windows completion ports and threadpool I/O facilities, it doesn't matter if one thread blocks every so often. Windows will detect this and schedule another one to run, such that there is one active thread scheduled on every core. I/O completion ports facilitate thread-agnostic I/O completion; the thread that blocked because it was extending an encrypted file won't impede the latency of other clients because there is no thread-specific association. I exploit all of this with PyParallel. http://pyparallel.org http://pyparallel.org For more details on asynchronous I/O on Windows and why it's fundamentally better than the UNIX I/O model in every possible way: https://speakerdeck.com/trent/pyparallel-how-we-removed-the-gil-and-exploited-all-cores?slide=47 https://speakerdeck.com/trent/pyparallel-how-we-removed-the-...
trentnelson 11y ago
> Well, apparently not on Windows. Agreed! http://pyparallel.org http://pyparallel.org > But this is a kernel interface issue not a language issue, Absolutely agree! The thing that surprises me most about Rust and Go in particular is that they're trying to solve an OS problem with new language abstractions. You don't need a new language -- what you really want is thread-agnostic I/O primitives, thread-agnostic "threadpool" primitives, and completion-oriented APIs that provide both synchronous and asynchronous primitives. Basically, you want what Windows has.