5 ms·

Linux is the gold standard for (1)? Windows NT 3.5 (1994) would beg to differ with its support for IO completion ports (IOCP). I think io_uring is more general

Linux is the gold standard for (1)? Windows NT 3.5 (1994) would beg to differ with its support for IO completion ports (IOCP). I think io_uring is more general and more flexible for the sort of M:N scheduling systems now commonly used in programming languages, but IOCP predates io_uring by 25 years! The rest of your comment sounds right to me, there are unanswered questions about how DirectStorage interacts with filesystem filter drivers.

wtallis 6y ago
Being old isn't particularly interesting here. io_uring can do 2.5M IOPS per CPU core with a sufficiently fast SSD. I've never seen any suggestion that IO overhead on Windows can be cut down enough to get close to that. And if IOCP was actually good for this use case, Microsoft wouldn't be hyping a new DirectStorage API. (It might turn out that DirectStorage is new window dressing on top of IOCP, but I suspect they've had to cut stuff out of the existing IO stack to reduce overhead.)
- AaronFriel 6y ago
  I think my point was largely that Windows already has (1) from your list and it's already the gold standard on Windows, and has been available for some 25 years. I would be surprised if it wasn't part of the mechanism used in DirectStorage. io_uring does have a more ergonomic and POSIX API, but that's because it's on Linux! Fundamentally they're both completion port based replacements to polling for IO events and they're both capable of high throughput and horizontal scaling.
  - wtallis 6y ago
    Ok, the misunderstanding here may be that I was using "gold standard" as a superlative among async IO APIs in general, while you're using it as a superlative among a given platform's set of IO APIs, async or otherwise.
PeCaN 6y ago
I was extremely (I think justifiably) skeptical of async I/O on Linux for the longest time but now I legitimately think with io_uring Linux is actually somehow the gold standard for (1). They really got their act together. io_uring is really good.

wtallis 6y ago

Being old isn't particularly interesting here. io_uring can do 2.5M IOPS per CPU core with a sufficiently fast SSD. I've never seen any suggestion that IO overhead on Windows can be cut down enough to get close to that. And if IOCP was actually good for this use case, Microsoft wouldn't be hyping a new DirectStorage API. (It might turn out that DirectStorage is new window dressing on top of IOCP, but I suspect they've had to cut stuff out of the existing IO stack to reduce overhead.)

AaronFriel 6y ago
I think my point was largely that Windows already has (1) from your list and it's already the gold standard on Windows, and has been available for some 25 years. I would be surprised if it wasn't part of the mechanism used in DirectStorage. io_uring does have a more ergonomic and POSIX API, but that's because it's on Linux! Fundamentally they're both completion port based replacements to polling for IO events and they're both capable of high throughput and horizontal scaling.
- wtallis 6y ago
  Ok, the misunderstanding here may be that I was using "gold standard" as a superlative among async IO APIs in general, while you're using it as a superlative among a given platform's set of IO APIs, async or otherwise.

PeCaN 6y ago

I was extremely (I think justifiably) skeptical of async I/O on Linux for the longest time but now I legitimately think with io_uring Linux is actually somehow the gold standard for (1). They really got their act together. io_uring is really good.