4 ms·

Alex Crichton started with a completion based Future struct in 2015. It was even (unstable) in std in 1.0.0: https://doc.rust-lang.org/1.0.0/std/sync/struct.Fu

Alex Crichton started with a completion based Future struct in 2015. It was even (unstable) in std in 1.0.0: https://doc.rust-lang.org/1.0.0/std/sync/struct.Future.html https://doc.rust-lang.org/1.0.0/std/sync/struct.Future.html Our async IO model was based on the Linux industry standard (then and now) epoll, but that is not at all what drove the switch to a polling based model, and the polling based model presents no issues whatsoever with io-uring. You do not know what you are talking about.

newpavlov 6y ago
>Our async IO model was based on the Linux industry standard (then and now) epoll, but that is not at all what drove the switch to a polling based model Can you provide a link to a design document or at the very least to a discussion with motivation for this switch outside of the desire to be as compatible as possible with the "Linux industry standard"? >the polling based model presents no issues whatsoever with io-uring There are no issues with io-uring compatibility to such extent that you wrote about a whole blog post about those issues: https://boats.gitlab.io/blog/post/io-uring/ https://boats.gitlab.io/blog/post/io-uring/ IIUC the best solutions right now are either to copy data around (bye-bye zero-cost) or to use another Pin-like awkward hack with executor-based buffer management, instead of using simple and familiar buffers which are part of a future state.
- withoutboats2 6y ago
  https://aturon.github.io/blog/2016/09/07/futures-design/ https://aturon.github.io/blog/2016/09/07/futures-design/ The completion based futures that Alex started with were also based on epoll. The performance issues it presented had nothing to do any sort of impedence mismatch between a completion based future and epoll, because there is no impedence issue. You are confused.
  - newpavlov 6y ago
    Thank you for the link! But immideately we can see the false equivalence: completion based API does not imply the callback-based approach. The article critigues the latter, but not the former. Earlier in this thread I've described how I see a completion-based model built on top of FSMs generated by compiler from async fns. In other words, the arguments presented in that article do not apply to this discussion. >The performance issues it presented had nothing to do any sort of impedence mismatch between a completion based future and epoll Sorry, but what? Even aturon's article states zero-cost as one of the 3 main goals. So performance issues with strong roots in the selected model is a very big problem in my book. >You do not know what you are talking about. >You are confused. Please, tone down your replies.
    - nemothekid 6y ago
      >Earlier in this thread I've described how I see a completion-based model built on top of FSMs generated by compiler from async fns. In other words, the arguments presented in that article do not apply to this discussion. I've been lurking your responses, but now I'm confused. If you are not using a callback based approach, then what are you using? Rust's FSM approach is predicated on polling; In other words if you aren't using callbacks, then how do you know that Future A has finished? If the answer is to use Rust's current systems, then that means the FSM is "polled" periodically, and then you still have "async Drop" problem as described in withoutboat's notorious article and furthermore, you haven't really changed Rust's design. Edit: As I've seen you mention in other threads, you need a sound design for async Drop for this to work. I'm not sure this is possible in Rust 1.0 (as Drop isn't currently required to run in safe Rust). That said it's unfair to call async "rushed", when your proposed design wouldn't even work in Rust 1.0. I'd be hesitant to call the design of the entire language rushed just because it didn't include linear types.
      - newpavlov 6y ago
        I meant the callback based approach described in the article, for example take this line from it: >Unfortunately, this approach nevertheless forces allocation at almost every point of future composition, and often imposes dynamic dispatch, despite our best efforts to avoid such overhead. It clearly does not apply to the model which I've described earlier. Of course, the described FSM state transition functions can be rightfully called callbacks, which adds a certain amount of confusion. I can agree with the argument that a proper async Drop can not be implemented in Rust 1.0, so we have to settle with a compromise solution. Same with proper self-referential structs vs Pin. But I would like to see this argument to be explicitly stated with sufficient backing of the impossibility statements.
        nemothekid 6y ago
        >Of course, the described FSM state transition functions can be rightfully called callbacks, which adds a certain amount of confusion. No, I'm not talking about the state transition functions. I'm talking about the runtime - the thing that will call the state transition function. In the current design, abstractly, the runtime polls/checks every if future if it's in a runnable state, and if so executes it. In an completion based design the future itself tells the runtime that the value is ready (either driven by a kernel thread, another thread or some other callback). (conceptually the difference is, in an poll based design, the future calls waker.wake(), and in a completion one, the future just calls the callback fn). Aaron has already described why that is a problem. The confusion I have is that both would have problems integrating io_uring into rust (due to the Drop problem; as Rust has no concept of the kernel owning a buffer), but your proposed solution seems strictly worse as it requires async Drop to be sound which is not guaranteed by Rust; which would make it useless for programs that are being written today. As a result, I'm having trouble accepting that your criticism is actually valid - what you seem to be arguing is that async/await should have never been stabilized in Rust 1.0, which I believe is a fair criticism, but it isn't one that indicates that the current design has been rushed. Upon further thought, I think your design ultimately requires futures to be implemented as a language feature, rather than a library (ex. for the future itself to expose multiple state transition functions without allocating is not possible with the current Trait system), which wouldn't have worked without forking Rust during the prototype stage.

newpavlov 6y ago

>Our async IO model was based on the Linux industry standard (then and now) epoll, but that is not at all what drove the switch to a polling based model Can you provide a link to a design document or at the very least to a discussion with motivation for this switch outside of the desire to be as compatible as possible with the "Linux industry standard"? >the polling based model presents no issues whatsoever with io-uring There are no issues with io-uring compatibility to such extent that you wrote about a whole blog post about those issues: https://boats.gitlab.io/blog/post/io-uring/ https://boats.gitlab.io/blog/post/io-uring/ IIUC the best solutions right now are either to copy data around (bye-bye zero-cost) or to use another Pin-like awkward hack with executor-based buffer management, instead of using simple and familiar buffers which are part of a future state.

withoutboats2 6y ago
https://aturon.github.io/blog/2016/09/07/futures-design/ https://aturon.github.io/blog/2016/09/07/futures-design/ The completion based futures that Alex started with were also based on epoll. The performance issues it presented had nothing to do any sort of impedence mismatch between a completion based future and epoll, because there is no impedence issue. You are confused.
- newpavlov 6y ago
  Thank you for the link! But immideately we can see the false equivalence: completion based API does not imply the callback-based approach. The article critigues the latter, but not the former. Earlier in this thread I've described how I see a completion-based model built on top of FSMs generated by compiler from async fns. In other words, the arguments presented in that article do not apply to this discussion. >The performance issues it presented had nothing to do any sort of impedence mismatch between a completion based future and epoll Sorry, but what? Even aturon's article states zero-cost as one of the 3 main goals. So performance issues with strong roots in the selected model is a very big problem in my book. >You do not know what you are talking about. >You are confused. Please, tone down your replies.
  - nemothekid 6y ago
    >Earlier in this thread I've described how I see a completion-based model built on top of FSMs generated by compiler from async fns. In other words, the arguments presented in that article do not apply to this discussion. I've been lurking your responses, but now I'm confused. If you are not using a callback based approach, then what are you using? Rust's FSM approach is predicated on polling; In other words if you aren't using callbacks, then how do you know that Future A has finished? If the answer is to use Rust's current systems, then that means the FSM is "polled" periodically, and then you still have "async Drop" problem as described in withoutboat's notorious article and furthermore, you haven't really changed Rust's design. Edit: As I've seen you mention in other threads, you need a sound design for async Drop for this to work. I'm not sure this is possible in Rust 1.0 (as Drop isn't currently required to run in safe Rust). That said it's unfair to call async "rushed", when your proposed design wouldn't even work in Rust 1.0. I'd be hesitant to call the design of the entire language rushed just because it didn't include linear types.
    - newpavlov 6y ago
      I meant the callback based approach described in the article, for example take this line from it: >Unfortunately, this approach nevertheless forces allocation at almost every point of future composition, and often imposes dynamic dispatch, despite our best efforts to avoid such overhead. It clearly does not apply to the model which I've described earlier. Of course, the described FSM state transition functions can be rightfully called callbacks, which adds a certain amount of confusion. I can agree with the argument that a proper async Drop can not be implemented in Rust 1.0, so we have to settle with a compromise solution. Same with proper self-referential structs vs Pin. But I would like to see this argument to be explicitly stated with sufficient backing of the impossibility statements.
      - nemothekid 6y ago
        >Of course, the described FSM state transition functions can be rightfully called callbacks, which adds a certain amount of confusion. No, I'm not talking about the state transition functions. I'm talking about the runtime - the thing that will call the state transition function. In the current design, abstractly, the runtime polls/checks every if future if it's in a runnable state, and if so executes it. In an completion based design the future itself tells the runtime that the value is ready (either driven by a kernel thread, another thread or some other callback). (conceptually the difference is, in an poll based design, the future calls waker.wake(), and in a completion one, the future just calls the callback fn). Aaron has already described why that is a problem. The confusion I have is that both would have problems integrating io_uring into rust (due to the Drop problem; as Rust has no concept of the kernel owning a buffer), but your proposed solution seems strictly worse as it requires async Drop to be sound which is not guaranteed by Rust; which would make it useless for programs that are being written today. As a result, I'm having trouble accepting that your criticism is actually valid - what you seem to be arguing is that async/await should have never been stabilized in Rust 1.0, which I believe is a fair criticism, but it isn't one that indicates that the current design has been rushed. Upon further thought, I think your design ultimately requires futures to be implemented as a language feature, rather than a library (ex. for the future itself to expose multiple state transition functions without allocating is not possible with the current Trait system), which wouldn't have worked without forking Rust during the prototype stage.