4 ms·

Most of the work you're talking about has been completed for a long time in the crate "mio". It abstracts out all of the platform dependent async io operations

by bvinc 7y ago

Most of the work you're talking about has been completed for a long time in the crate "mio". It abstracts out all of the platform dependent async io operations into a single consistent interface.

Const-me 7y ago
Are you sure about that? I've looked at the library, it doesn't support anything besides epoll and IOCP, i.e. no BSD or OSX support, no AIO on Linux (the kernel one, not POSIX). You probably don't want to consume IOCP API on Windows anymore, Vista introduced higher level and easier to use StartThreadpoolIo and friends. Also on Windows, even with IOCP, you don't want to split async IO from thread pool. The OS kernel manages them both at the same time. Work stealing or other custom scheduling is usually slower than what kernel does.
- bvinc 7y ago
  It says that it is also backed by kqueue and supports FreeBSD , NetBSD, and OS X. I found this issue where they're discussing rewriting the Windows implementation using the library `wepoll`. https://github.com/tokio-rs/gsoc/issues/3 https://github.com/tokio-rs/gsoc/issues/3 What would be the advantages of supporting AIO on Linux?
  - Const-me 7y ago
    > rewriting the Windows implementation using the library `wepoll`. Interesting idea but I don’t like it too much. While removing the huge complexity of manually managing IOCP and required resources, It’s an undocumented API. New Vista+ threadpool-based IO also removes that complexity, removes complexity of implementing thread pools in the higher level in Tokyo. It’s documented and supported, and Rust has issues with WinXP support anyway. > What would be the advantages of supporting AIO on Linux? Faster async disk IO for the apps which are OK with the limitations, i.e. which read/write complete blocks of O_DIRECT files. Databases come to mind. BTW, io_uring feature coming into Linux kernel removes most limitations of AIO while also improving performance.

by bvinc 7y ago

Most of the work you're talking about has been completed for a long time in the crate "mio". It abstracts out all of the platform dependent async io operations into a single consistent interface.

Const-me 7y ago

Are you sure about that? I've looked at the library, it doesn't support anything besides epoll and IOCP, i.e. no BSD or OSX support, no AIO on Linux (the kernel one, not POSIX). You probably don't want to consume IOCP API on Windows anymore, Vista introduced higher level and easier to use StartThreadpoolIo and friends. Also on Windows, even with IOCP, you don't want to split async IO from thread pool. The OS kernel manages them both at the same time. Work stealing or other custom scheduling is usually slower than what kernel does.

bvinc 7y ago
It says that it is also backed by kqueue and supports FreeBSD , NetBSD, and OS X. I found this issue where they're discussing rewriting the Windows implementation using the library `wepoll`. https://github.com/tokio-rs/gsoc/issues/3 https://github.com/tokio-rs/gsoc/issues/3 What would be the advantages of supporting AIO on Linux?
- Const-me 7y ago
  > rewriting the Windows implementation using the library `wepoll`. Interesting idea but I don’t like it too much. While removing the huge complexity of manually managing IOCP and required resources, It’s an undocumented API. New Vista+ threadpool-based IO also removes that complexity, removes complexity of implementing thread pools in the higher level in Tokyo. It’s documented and supported, and Rust has issues with WinXP support anyway. > What would be the advantages of supporting AIO on Linux? Faster async disk IO for the apps which are OK with the limitations, i.e. which read/write complete blocks of O_DIRECT files. Databases come to mind. BTW, io_uring feature coming into Linux kernel removes most limitations of AIO while also improving performance.