4 ms·

This aspect of io_uring does affect a lot of surface APIs, as I have experienced at work. At least for me I didn't have to worry much about borrowing though.

eqvinox 2y ago
Hmm. Does it? Python's futures have an explicit .cancel() operation. And the C io_uring usage I'm looking at knows to cancel events too… It's really that Rust might've made a poor choice here, as the article points out: > Async Rust makes a few core assumptions about futures: > 1. The state of futures only change when they are polled. > 2. Futures are implicitly cancellable by simply never polling them again. But at the same time, maybe this is just that Rust's Futures need to be used different here, in conjunction with a separate mechanism to manage the I/O operation that knows things need to be cancelled?
- lifthrasiir 2y ago
  That part of the article is kinda irrelevant in my opinion. Futures do require polling to move forward, but polling can be forced by an external signal (otherwise the whole future model wouldn't work!). Therefore io_uring can be safely implemented by having central worker threads which then signal outstanding futures; that was how I ended up doing at work as well. So the article actually seems to ask whether such out-of-band mechanism can be entirely prevented or not.
  - eqvinox 2y ago
    The sibling comment to yours points out cancelling Futures is dropping Futures, what's your experience / do you think that would work to prevent needing the out-of-band mechanism?
    - lifthrasiir 2y ago
      That's backwards... Rust's way to do cancellation is simply to drop the future (i.e. let it deallocated). There is one big caveat here though, namely the lack of async drop as others pointed out. In the current Rust io_uring-like stuffs can be safely implemented with an additional layer of abstraction. Some io_uring operations can be ongoing when it looks fine to borrow the buffer, sure. Your API just has to ensure that it is not possible to borrow until all operations are finished then! Maybe it can error, or you can require something like `buf.borrow().await`. This explicit borrowing is not an alien concept in Rust (cf. RefCell etc.) and probably the best design at the moment, but it does need dynamic bookkeeping which some may want to eliminate.
- eptcyka 2y ago
  The assumption made by rust is that a future is cancelled when it is dropped.
  - eqvinox 2y ago
    Ah, Thanks, that makes sense, but then I don't understand how this isn't just a bug in these Rust runtimes. As in: the drop codepath on the future needs to not only submit the cancellation SQE into io_uring, it also needs to still process CQEs from the original request that pop up before the CQE for the cancellation… NB: I have only done a little bit of Rust, but am hoping to move there in the future — but I am working on C code interfacing io_uring… I will say doing this correctly does in fact require a bit of brainpower when writing that code.
    - eptcyka 2y ago
      I am not well versed in the async things as of late, but one complication is that the drop implementation is a blocking one. This could easily lead to deadlocks. Or the drop implementation could spawn an async task to clean up after itself later.
- fnord123 2y ago
  > First of all, we are fortunate that the I/O safety problem can be addressed now, which safe Rust aims to ensure this in the future. Rust provides the Drop trait to define custom behavior when a value is cleaned up. Thus, we can do something like this... > We just need to encourage async runtimes to implement this fix. This likely needs async drop if you need to perform a follow up call to cancel the outstanding tasks or closing the open sockets. Async Drop is currently experimental: https://github.com/rust-lang/compiler-team/issues/727 https://github.com/rust-lang/compiler-team/issues/727

eqvinox 2y ago

Hmm. Does it? Python's futures have an explicit .cancel() operation. And the C io_uring usage I'm looking at knows to cancel events too… It's really that Rust might've made a poor choice here, as the article points out: > Async Rust makes a few core assumptions about futures: > 1. The state of futures only change when they are polled. > 2. Futures are implicitly cancellable by simply never polling them again. But at the same time, maybe this is just that Rust's Futures need to be used different here, in conjunction with a separate mechanism to manage the I/O operation that knows things need to be cancelled?

lifthrasiir 2y ago
That part of the article is kinda irrelevant in my opinion. Futures do require polling to move forward, but polling can be forced by an external signal (otherwise the whole future model wouldn't work!). Therefore io_uring can be safely implemented by having central worker threads which then signal outstanding futures; that was how I ended up doing at work as well. So the article actually seems to ask whether such out-of-band mechanism can be entirely prevented or not.
- eqvinox 2y ago
  The sibling comment to yours points out cancelling Futures is dropping Futures, what's your experience / do you think that would work to prevent needing the out-of-band mechanism?
  - lifthrasiir 2y ago
    That's backwards... Rust's way to do cancellation is simply to drop the future (i.e. let it deallocated). There is one big caveat here though, namely the lack of async drop as others pointed out. In the current Rust io_uring-like stuffs can be safely implemented with an additional layer of abstraction. Some io_uring operations can be ongoing when it looks fine to borrow the buffer, sure. Your API just has to ensure that it is not possible to borrow until all operations are finished then! Maybe it can error, or you can require something like `buf.borrow().await`. This explicit borrowing is not an alien concept in Rust (cf. RefCell etc.) and probably the best design at the moment, but it does need dynamic bookkeeping which some may want to eliminate.
eptcyka 2y ago
The assumption made by rust is that a future is cancelled when it is dropped.
- eqvinox 2y ago
  Ah, Thanks, that makes sense, but then I don't understand how this isn't just a bug in these Rust runtimes. As in: the drop codepath on the future needs to not only submit the cancellation SQE into io_uring, it also needs to still process CQEs from the original request that pop up before the CQE for the cancellation… NB: I have only done a little bit of Rust, but am hoping to move there in the future — but I am working on C code interfacing io_uring… I will say doing this correctly does in fact require a bit of brainpower when writing that code.
  - eptcyka 2y ago
    I am not well versed in the async things as of late, but one complication is that the drop implementation is a blocking one. This could easily lead to deadlocks. Or the drop implementation could spawn an async task to clean up after itself later.
fnord123 2y ago
> First of all, we are fortunate that the I/O safety problem can be addressed now, which safe Rust aims to ensure this in the future. Rust provides the Drop trait to define custom behavior when a value is cleaned up. Thus, we can do something like this... > We just need to encourage async runtimes to implement this fix. This likely needs async drop if you need to perform a follow up call to cancel the outstanding tasks or closing the open sockets. Async Drop is currently experimental: https://github.com/rust-lang/compiler-team/issues/727 https://github.com/rust-lang/compiler-team/issues/727