3 ms·

C++20 coroutines and io_uring

67 points by signa11 4y ago

max_k 4y ago
I love C++ coroutines! (The spec has a few warts and gives me enough reasons to hate them, like not being able by definition to properly inline nested coroutine calls, but I love them anyway.) I've written several libraries for integrating C++ coroutines with stuff like io_uring, libcurl, c_ares, libpq and more. For example, this is how using my io_uring/coroutine library can be used: https://github.com/CM4all/libcommon/blob/master/test/co/RunCoReadTextFile.cxx https://github.com/CM4all/libcommon/blob/master/test/co/RunC... auto result = co_await CoReadTextFile(queue, AT_FDCWD, path); co_await CoWrite(queue, STDOUT_FILENO, result.data(), result.size(), 0); This opens a file, stats it, reads its contents, and writes it to stdout - all 4 I/O operations are asynchronous with io_uring. Source code for CoReadTextFile() which is also a coroutine: https://github.com/CM4all/libcommon/blob/master/src/io/uring/CoTextFile.cxx https://github.com/CM4all/libcommon/blob/master/src/io/uring... Sample code for libpq: https://github.com/CM4all/libcommon/blob/master/test/co/RunCoQuery.cxx https://github.com/CM4all/libcommon/blob/master/test/co/RunC... and c_ares https://github.com/CM4all/libcommon/blob/master/test/co/RunCoLookup.cxx https://github.com/CM4all/libcommon/blob/master/test/co/RunC... and libcurl https://github.com/CM4all/libcommon/blob/master/test/curl/RunCoHttpGet.cxx https://github.com/CM4all/libcommon/blob/master/test/curl/Ru... I wrote all of this for proprietary applications at dayjob, but the core library is open source, as is much of my dayjob code. The I/O event loop this integrates with is also used by several open source projects I maintain, e.g. the Music Player Daemon (https://github.com/MusicPlayerDaemon/MPD/tree/master/src/event/ https://github.com/MusicPlayerDaemon/MPD/tree/master/src/eve...) which can also take advantage of io_uring, though not (yet) with coroutines, only "classic" non-blocking I/O. My code is optimized for low-overhead; the very core doesn't even use std::function because I fear its implicit heap allocations. Long ago, I used boost::asio (which also integrates well with coroutines) but didn't like it because it was too bloated for me. I've rarely seen other nerds talk about C++ coroutines, and never about integrating them with io_uring, made me thinking I'm the only one. But maybe all the others just don't write/blog about it - I never did either... That's why this blog was a refreshing read for me, thanks.
- pencilguin 4y ago
  Others have integrated io_uring with asio, and asio with co-routines. The performance is about the same. The complexity is a matter of taste: asio offers lots built in that you would need to code yourself for the next project. I generally lean toward lower total complexity at cost if more custom coding, but it is a big tent.
  - wolletd 4y ago
    I recently built something to connect asio with gRPC: https://github.com/WolleTD/asio-grpc-callback https://github.com/WolleTD/asio-grpc-callback The channels and co_yield-capable coros from asio::experimental are quite fun to use for streams. When you have prior experience with async programming in other languages, e.g. Python or Rust, it's really handy that you don't have to define any awaitable types to use asio with coroutines.
- mrfox321 4y ago
  All new c++ code I work with at bigco are c++ coroutines. The underlying executors come from folly.
- intelVISA 4y ago
  We have an eerily similar coding style... I am now unnerved.
  - max_k 4y ago
    Whoa, that exists? How about having some nerd fun and we create a PR for each other's open source project?
    - intelVISA 4y ago
      Interesting... mail me? I will meditate on this.
- ptr 4y ago
  How do you deal with errors? Do you check them at every callsite and manually “bubble up”, or do you use exceptions? I’d love to use coroutines but we don’t use exceptions and I’m not super excited about handling errors manually.
gavinray 4y ago
Hey just a heads up -- you're setting the queue depth of the ring to the number of files discovered, that's typically not the way you'd want to size the ring. When you start to get up into higher queue depths, like 512-1,024, you benefit a lot from using flags like IORING_SETUP_SQPOLL. If you're going to do a thread pool too, you may as well use IORING_WQ_ATTACH and put a ring on each thread that shares a single kernel poller, see the SO explanation I gave recently here: https://stackoverflow.com/questions/73798651/how-to-create-a-single-sqpoll-thread-in-io-uring-for-multiple-rings-ioring-setu/74256951#74256951 https://stackoverflow.com/questions/73798651/how-to-create-a...

67 points by signa11 4y ago

I love C++ coroutines! (The spec has a few warts and gives me enough reasons to hate them, like not being able by definition to properly inline nested coroutine calls, but I love them anyway.) I've written several libraries for integrating C++ coroutines with stuff like io_uring, libcurl, c_ares, libpq and more. For example, this is how using my io_uring/coroutine library can be used: https://github.com/CM4all/libcommon/blob/master/test/co/RunCoReadTextFile.cxx https://github.com/CM4all/libcommon/blob/master/test/co/RunC... auto result = co_await CoReadTextFile(queue, AT_FDCWD, path); co_await CoWrite(queue, STDOUT_FILENO, result.data(), result.size(), 0); This opens a file, stats it, reads its contents, and writes it to stdout - all 4 I/O operations are asynchronous with io_uring. Source code for CoReadTextFile() which is also a coroutine: https://github.com/CM4all/libcommon/blob/master/src/io/uring/CoTextFile.cxx https://github.com/CM4all/libcommon/blob/master/src/io/uring... Sample code for libpq: https://github.com/CM4all/libcommon/blob/master/test/co/RunCoQuery.cxx https://github.com/CM4all/libcommon/blob/master/test/co/RunC... and c_ares https://github.com/CM4all/libcommon/blob/master/test/co/RunCoLookup.cxx https://github.com/CM4all/libcommon/blob/master/test/co/RunC... and libcurl https://github.com/CM4all/libcommon/blob/master/test/curl/RunCoHttpGet.cxx https://github.com/CM4all/libcommon/blob/master/test/curl/Ru... I wrote all of this for proprietary applications at dayjob, but the core library is open source, as is much of my dayjob code. The I/O event loop this integrates with is also used by several open source projects I maintain, e.g. the Music Player Daemon (https://github.com/MusicPlayerDaemon/MPD/tree/master/src/event/ https://github.com/MusicPlayerDaemon/MPD/tree/master/src/eve...) which can also take advantage of io_uring, though not (yet) with coroutines, only "classic" non-blocking I/O. My code is optimized for low-overhead; the very core doesn't even use std::function because I fear its implicit heap allocations. Long ago, I used boost::asio (which also integrates well with coroutines) but didn't like it because it was too bloated for me. I've rarely seen other nerds talk about C++ coroutines, and never about integrating them with io_uring, made me thinking I'm the only one. But maybe all the others just don't write/blog about it - I never did either... That's why this blog was a refreshing read for me, thanks.

pencilguin 4y ago
Others have integrated io_uring with asio, and asio with co-routines. The performance is about the same. The complexity is a matter of taste: asio offers lots built in that you would need to code yourself for the next project. I generally lean toward lower total complexity at cost if more custom coding, but it is a big tent.
- wolletd 4y ago
  I recently built something to connect asio with gRPC: https://github.com/WolleTD/asio-grpc-callback https://github.com/WolleTD/asio-grpc-callback The channels and co_yield-capable coros from asio::experimental are quite fun to use for streams. When you have prior experience with async programming in other languages, e.g. Python or Rust, it's really handy that you don't have to define any awaitable types to use asio with coroutines.
mrfox321 4y ago
All new c++ code I work with at bigco are c++ coroutines. The underlying executors come from folly.
intelVISA 4y ago
We have an eerily similar coding style... I am now unnerved.
- max_k 4y ago
  Whoa, that exists? How about having some nerd fun and we create a PR for each other's open source project?
  - intelVISA 4y ago
    Interesting... mail me? I will meditate on this.
ptr 4y ago
How do you deal with errors? Do you check them at every callsite and manually “bubble up”, or do you use exceptions? I’d love to use coroutines but we don’t use exceptions and I’m not super excited about handling errors manually.

gavinray 4y ago

Hey just a heads up -- you're setting the queue depth of the ring to the number of files discovered, that's typically not the way you'd want to size the ring. When you start to get up into higher queue depths, like 512-1,024, you benefit a lot from using flags like IORING_SETUP_SQPOLL. If you're going to do a thread pool too, you may as well use IORING_WQ_ATTACH and put a ring on each thread that shares a single kernel poller, see the SO explanation I gave recently here: https://stackoverflow.com/questions/73798651/how-to-create-a-single-sqpoll-thread-in-io-uring-for-multiple-rings-ioring-setu/74256951#74256951 https://stackoverflow.com/questions/73798651/how-to-create-a...