4 ms·

That's an excellent document, thanks for writing it up! One motivation for completion futures that either wasn't mentioned or is an unexpected side effect of p

That's an excellent document, thanks for writing it up! One motivation for completion futures that either wasn't mentioned or is an unexpected side effect of point #1 (compatibility with C++) is that async code in C/C++ can take non-owning references into buffers. For example, on Linux if you issue an async read() call using io_uring and you later get cancelled, you have to tell the kernel somehow that it needs to not touch the buffer after Rust frees it. There are ways to do this, such as giving the kernel ownership of the buffers using IORING_REGISTER_BUFFERS, but having the kernel own the I/O buffers can make things awkward. (Async C++ folks have shown me patterns that would require copies in this case.) Have you all given any thought as to the best practices here? It's a tough decision, as the only real solution I can think of involves making poll() unsafe, which is unsavory (NB: not necessarily wrong).

Arnavion 5y ago
We have prior art of the rio crate where the Future is responsible for keeping the buffer alive until the kernel is done with it, which it does by blocking in its Drop. But since futures can be std::mem::forget'd that doesn't work (to put it charitably), and is why rio warns its users not to do that. AFAIK there's no better way than to have the library that wraps io_uring also own the buffers instead of letting the user own them, so that it can also control when / if they're dropped.
- pcwalton 5y ago
  The way to make that work would be to make poll() unsafe and require async runtimes to uphold the invariant that poll() must be called until completion if Pending is ever returned. That way, runtimes would ensure memory safety. This doesn't mean that functions using async/await would have to be unsafe, just functions that manually implement poll. I still don't love that, though.
  - Arnavion 5y ago
    There's no way for an async runtime to do such a thing. An async runtime only knows about the top-level Future of each task. It knows nothing about what other Futures' poll()s the top-level Future's poll() calls in turn. Async-await does not obviate this problem. `let (value, _) = futures::future::select(f1, f2).await.factor_first();` has the same problem, one of the futures is going to be dropped before it resolves.
    - pcwalton 5y ago
      > It knows nothing about what other Futures' poll()s the top-level Future's poll() calls in turn. That's why calling poll would be marked unsafe. When you call poll() manually it's on you to uphold the invariants. > Async-await does not obviate this problem. `let (value, _) = futures::future::select(f1, f2).await.factor_first();` has the same problem, one of the futures is going to be dropped before it resolves. Yes, you also need to reform select. Matthias247 has done work to flesh out what this looks like: https://github.com/Matthias247/rfcs/pull/1 https://github.com/Matthias247/rfcs/pull/1
      - Arnavion 5y ago
        >Matthias247 has done work to flesh out what this looks like: https://github.com/Matthias247/rfcs/pull/1 https://github.com/Matthias247/rfcs/pull/1 By making a new kind of async function that cannot be called from existing async functions, I see. I started reading it with the expectation that RTCFutures would automatically spawn() themselves to enforce the RTC requirement instead of introducing a third function color. I'm pessimistic a third function color will catch on, but let's see how it goes.
yxhuvud 5y ago
The cancel operation in io_uring returns a status when it completes, which mean the memory can't be freed until the ring acknowledges the cancellation. That accomplishes telling the kernel that it needs not touch the referred to memory. This follows the normal uring operation and mean that cancellation of any outstanding io in itself would need to be waited upon. There is also need to be able to handle the race condition where the task completes successfully during cancellation in any case, or else you will get completions which you don't know to handle. So the support for cancellation must already handle processing of the completion events until the cancellation is acknowledged, even if the userspace submission of the cancellation happened long ago. This mean io cancellation will be fairly cheap but not necessarily totally wait free. Is that really that big of a problem? It seems you make a lot of problems for yourself if waiting for acknowledgement isn't good enough.

Arnavion 5y ago

We have prior art of the rio crate where the Future is responsible for keeping the buffer alive until the kernel is done with it, which it does by blocking in its Drop. But since futures can be std::mem::forget'd that doesn't work (to put it charitably), and is why rio warns its users not to do that. AFAIK there's no better way than to have the library that wraps io_uring also own the buffers instead of letting the user own them, so that it can also control when / if they're dropped.

pcwalton 5y ago
The way to make that work would be to make poll() unsafe and require async runtimes to uphold the invariant that poll() must be called until completion if Pending is ever returned. That way, runtimes would ensure memory safety. This doesn't mean that functions using async/await would have to be unsafe, just functions that manually implement poll. I still don't love that, though.
- Arnavion 5y ago
  There's no way for an async runtime to do such a thing. An async runtime only knows about the top-level Future of each task. It knows nothing about what other Futures' poll()s the top-level Future's poll() calls in turn. Async-await does not obviate this problem. `let (value, _) = futures::future::select(f1, f2).await.factor_first();` has the same problem, one of the futures is going to be dropped before it resolves.
  - pcwalton 5y ago
    > It knows nothing about what other Futures' poll()s the top-level Future's poll() calls in turn. That's why calling poll would be marked unsafe. When you call poll() manually it's on you to uphold the invariants. > Async-await does not obviate this problem. `let (value, _) = futures::future::select(f1, f2).await.factor_first();` has the same problem, one of the futures is going to be dropped before it resolves. Yes, you also need to reform select. Matthias247 has done work to flesh out what this looks like: https://github.com/Matthias247/rfcs/pull/1 https://github.com/Matthias247/rfcs/pull/1
    - Arnavion 5y ago
      >Matthias247 has done work to flesh out what this looks like: https://github.com/Matthias247/rfcs/pull/1 https://github.com/Matthias247/rfcs/pull/1 By making a new kind of async function that cannot be called from existing async functions, I see. I started reading it with the expectation that RTCFutures would automatically spawn() themselves to enforce the RTC requirement instead of introducing a third function color. I'm pessimistic a third function color will catch on, but let's see how it goes.

yxhuvud 5y ago

The cancel operation in io_uring returns a status when it completes, which mean the memory can't be freed until the ring acknowledges the cancellation. That accomplishes telling the kernel that it needs not touch the referred to memory. This follows the normal uring operation and mean that cancellation of any outstanding io in itself would need to be waited upon. There is also need to be able to handle the race condition where the task completes successfully during cancellation in any case, or else you will get completions which you don't know to handle. So the support for cancellation must already handle processing of the completion events until the cancellation is acknowledged, even if the userspace submission of the cancellation happened long ago. This mean io cancellation will be fairly cheap but not necessarily totally wait free. Is that really that big of a problem? It seems you make a lot of problems for yourself if waiting for acknowledgement isn't good enough.