3 ms·
Despite various attempts at Linux async file APIs over the years, it turned out using a well-implemented thread pool for file I/O was the fastest way to do asyn
by jlokier 3y ago
Despite various attempts at Linux async file APIs over the years, it turned out using a well-implemented thread pool for file I/O was the fastest way to do async file I/O on Linux, in most cases. In cases where it wasn't fastest, it was still close.
A thread pool was also more reliable than the kernel APIs at ensuring overlapped I/O for throughout. libaio (kernel API) calls sometimes turned into blocking calls depending on filesystem internals, in a way that thread pools don't.
POSIX AIO, on Linux, just uses a userspace thread pool. It's implemented in libc and is not particularly fast. You can do better with your own thread pool.
Perhaps Linux threads got faster, faster than the APIs got better.
Even with io_uring, in my tests on fast NVMe storage RAIDs where it should make the most difference, I found io_uring wasn't noticably faster than a well-optimised (for Linux) thread pool.
io_uring can potentally adapt better to shared workload and different cache situations, because it has access to kernel information that userspace thread pools are not granted. But I was surprised to find no significant random-access I/O performance increase between my thread pool and io_uring, when I was trying to optimise both methods to get the best throughput out of a system.