3 ms·

Well, it's "about" async Rust and io-uring inasmuch as they represent incompatible paradigms. Rust assumes as part of its model that "state only changes when p

by ajross 2y ago

Well, it's "about" async Rust and io-uring inasmuch as they represent incompatible paradigms. Rust assumes as part of its model that "state only changes when polled". Which is to say, it's not really "async" at all (none of these libraries are), it's just a framework for suspending in-progress work until it's ready. But "it's ready" is still a synchronous operation. But io-uring is actually async. Your process memory state is being changed by the kernel at moments that have nothing to do with the instruction being executed by the Rust code.

withoutboats3 2y ago
[flagged]
- ethegwo 2y ago
  > Rust does not assume that state changes only when polled. I will replace to more exact description about this, thanks.
- ajross 2y ago
  What you're describing is a synchronous process, though! ("When a message is put..."). That's the disconnect in the linked article. Two different concepts of asynchrony: one has to do with multiple contexts changing state without warning, the other (what you describe) is about suspending threads contexts "until" something happens.
  - withoutboats3 2y ago
    [flagged]
    - biorach 2y ago
      > Again you are wrong. A forum full of people who just like to hear themselves talk. I guess it makes you feel good in some way? I think you're being unduly harsh here. There are a variety of voices here, of various levels of expertise. If someone says something you think is incorrect but it seems that they are speaking in good faith then the best way to handle the situation is to politely provide a correct explanation. If you really think they are in bad faith then calmly call them out on it and leave the conversation.
      - nemothekid 2y ago
        I've been following withoutboats for ~6 years and it really feels like his patience has completely evaporated. I get it though, he has been really in the weeds of Rust's async implementation and has argued endlessly with those who don't like the tradeoffs but only have a surface level understanding of the problem. I think I've read this exact convo maybe 20+ times among HN, Reddit, Github Issues and Twitter among various topics including but not limited to, async i/o, Pin, and cancellation.
        withoutboats3 2y ago
        I freely admit I’m frustrated by the discourse around async Rust! I’m also very frustrated because I feel I was iced out of the project for petty reasons to do with whom I’m friends with and the people who were supposed to take over my work have done a very poor job, hence the failure to ship much of value to users. What we shipped in 2019 was an MVP that was intended to be followed by several improvements in quick succession, which the Rust project is only now moving toward delivering. I’ve written about this extensively. My opinion is that async Rust is an incredible achievement, primarily not mine (among the people who deserve more credit than me are Alex Crichton, Carl Lerche, and Aaron Turon). My only really significant contributions were making it safe to use references in an async function and documenting how to interface with completion based APIs like io-uring correctly. So it is very frustrating to see the discourse focused on inaccurate statements about async Rust which I believe is the best system for async IO in any language and which just needs to be finished.
        biorach 2y ago
        > So it is very frustrating to see the discourse focused on inaccurate statements about async Rust > No, ajross is very confidently making false descriptions of how async Rust and io-using operate. This website favors people who sound right whether or not they are, because most readers are not well informed but have a ridiculous confidence that they can infer what is true based on the tone and language used by a commenter. I find this deplorable and think this website is a big part of why discourse around computer science is so ignorant, and I respond accordingly when someone confronts me with comments like this. They had an inaccurate (from your point of view) understanding. That's all. If they were wrong that's not a reason to attack them. If you think they were over-confident (personally I don't) that's still not a reason to attack them. Again, I think ajross set out their understanding in a clear and polite manner. You should correct them in a similar manner.
    - ajross 2y ago
      > If that task has cancelled its interest in this syscall, they should instead clean up the resources owned by that CQE. So, first: how is that not consistent with the contention that the bug is due to a collision in the meaning of "asynchronous"? You're describing, once more, a synchronous operation ("when ... cancel") on a data structure that doesn't support that ("the kernel writes ..." on its own schedule). And second: the English language text of your solution has race conditions. How do you prevent reading from the buffer after the beginning of "cancel" and before the "dispatch"? You need some locking in there, which you don't in general async code. Ergo it's a paradigm clash. Developers, you among them it seems, don't really understand the requirements of a truly async process and get confused trying to shoehorn it into a "callbacks with context switch" framework like rust async.
      - withoutboats3 2y ago
        [flagged]
        ajross 2y ago
        [flagged]
        withoutboats3 2y ago
        [flagged]
        blinzy 2y ago
        Unbelievable! how bloody rude can you be? To the moderators (dang), do people get to keep their account here just because they're a "famous" poster despite writing the way they're doing all over this post? I'm assuming other posters have been banned for substantially less aggressive behaviour...
      - skavi 2y ago
        > Developers, you among them it seems, don't really understand the requirements of a truly async process and get confused trying to shoehorn it into a "callbacks with context switch" framework like rust async. This is an odd thing to say about someone who has written a correct solution to the problem which triggered this discussion. Also, you really need to define what truly async means. Many layers of computing are async or not async depending on how you look at them.
    - throw10920 2y ago
      > Again you are wrong. A forum full of people who just like to hear themselves talk. I guess it makes you feel good in some way? This is why people don't like the Rust community.
ethegwo 2y ago
the post only talks about "future state", maybe I'm not clearly to point out this. with epoll, accept syscall and future state changing is happened in the same polling, which io_uring is not. Once accept syscall is complete, future has already advanced to complete, but actually it is not at that moment in the real world Rust.
- withoutboats3 2y ago
  It's true, there's a necessary layer of abstraction with io-uring that doesn't exist with epoll. With epoll, the reactor just maps FDs to Wakers, and then wakes whatever Waker is waiting on that FD. Then that task does the syscall. With io-uring, instead the reactor is reading completion events from a queue. It processes those events, sets some state, and then wakes those tasks. Those tasks find the result of the syscall in that state that the reactor set. This is the difference between readiness (epoll) and completion (io-uring): with readiness the task wakes when the syscall is ready to be performed without blocking, with completion the task wakes when the syscall is already complete. When a task loses interest in an event in epoll, all that happens is it gets "spuriously awoken," so it sees there's nothing for it to do and goes back to sleep. With io-uring, the reactor needs to do more: when a task has lost interest in an incomplete event, that task needs to set the reactor into a state where instead of waking it, it will clean up the resources owned by the completion event. In the case of accept, this means closing that FD. According to your post, monoio fails to do this, and just spuriously wakes up the task, leaking the resource. The only way this relates to Rust's async model is that all futures in Rust are cancellable, so the reactor needs to handle the possibility that interest in a syscall is cancelled or the reactor is incorrect. But its completely possible to implement an io-uring reactor correctly under Rust's async model, this is just a requirement to do so.
  - 0x457 2y ago
    > But its completely possible to implement an io-uring reactor correctly under Rust's async model, this is just a requirement to do so. I don't get why people say it's incompatible with rust when rust async libraries work IOCP, which follows the similar model as io-uring?

by ajross 2y ago

withoutboats3 2y ago

[flagged]

ethegwo 2y ago
> Rust does not assume that state changes only when polled. I will replace to more exact description about this, thanks.
ajross 2y ago
What you're describing is a synchronous process, though! ("When a message is put..."). That's the disconnect in the linked article. Two different concepts of asynchrony: one has to do with multiple contexts changing state without warning, the other (what you describe) is about suspending threads contexts "until" something happens.
- withoutboats3 2y ago
  [flagged]
  - biorach 2y ago
    > Again you are wrong. A forum full of people who just like to hear themselves talk. I guess it makes you feel good in some way? I think you're being unduly harsh here. There are a variety of voices here, of various levels of expertise. If someone says something you think is incorrect but it seems that they are speaking in good faith then the best way to handle the situation is to politely provide a correct explanation. If you really think they are in bad faith then calmly call them out on it and leave the conversation.
    - nemothekid 2y ago
      I've been following withoutboats for ~6 years and it really feels like his patience has completely evaporated. I get it though, he has been really in the weeds of Rust's async implementation and has argued endlessly with those who don't like the tradeoffs but only have a surface level understanding of the problem. I think I've read this exact convo maybe 20+ times among HN, Reddit, Github Issues and Twitter among various topics including but not limited to, async i/o, Pin, and cancellation.
      - withoutboats3 2y ago
        I freely admit I’m frustrated by the discourse around async Rust! I’m also very frustrated because I feel I was iced out of the project for petty reasons to do with whom I’m friends with and the people who were supposed to take over my work have done a very poor job, hence the failure to ship much of value to users. What we shipped in 2019 was an MVP that was intended to be followed by several improvements in quick succession, which the Rust project is only now moving toward delivering. I’ve written about this extensively. My opinion is that async Rust is an incredible achievement, primarily not mine (among the people who deserve more credit than me are Alex Crichton, Carl Lerche, and Aaron Turon). My only really significant contributions were making it safe to use references in an async function and documenting how to interface with completion based APIs like io-uring correctly. So it is very frustrating to see the discourse focused on inaccurate statements about async Rust which I believe is the best system for async IO in any language and which just needs to be finished.
        biorach 2y ago
        > So it is very frustrating to see the discourse focused on inaccurate statements about async Rust > No, ajross is very confidently making false descriptions of how async Rust and io-using operate. This website favors people who sound right whether or not they are, because most readers are not well informed but have a ridiculous confidence that they can infer what is true based on the tone and language used by a commenter. I find this deplorable and think this website is a big part of why discourse around computer science is so ignorant, and I respond accordingly when someone confronts me with comments like this. They had an inaccurate (from your point of view) understanding. That's all. If they were wrong that's not a reason to attack them. If you think they were over-confident (personally I don't) that's still not a reason to attack them. Again, I think ajross set out their understanding in a clear and polite manner. You should correct them in a similar manner.
  - ajross 2y ago
    > If that task has cancelled its interest in this syscall, they should instead clean up the resources owned by that CQE. So, first: how is that not consistent with the contention that the bug is due to a collision in the meaning of "asynchronous"? You're describing, once more, a synchronous operation ("when ... cancel") on a data structure that doesn't support that ("the kernel writes ..." on its own schedule). And second: the English language text of your solution has race conditions. How do you prevent reading from the buffer after the beginning of "cancel" and before the "dispatch"? You need some locking in there, which you don't in general async code. Ergo it's a paradigm clash. Developers, you among them it seems, don't really understand the requirements of a truly async process and get confused trying to shoehorn it into a "callbacks with context switch" framework like rust async.
    - withoutboats3 2y ago
      [flagged]
      - ajross 2y ago
        [flagged]
        withoutboats3 2y ago
        [flagged]
        blinzy 2y ago
        Unbelievable! how bloody rude can you be? To the moderators (dang), do people get to keep their account here just because they're a "famous" poster despite writing the way they're doing all over this post? I'm assuming other posters have been banned for substantially less aggressive behaviour...
    - skavi 2y ago
      > Developers, you among them it seems, don't really understand the requirements of a truly async process and get confused trying to shoehorn it into a "callbacks with context switch" framework like rust async. This is an odd thing to say about someone who has written a correct solution to the problem which triggered this discussion. Also, you really need to define what truly async means. Many layers of computing are async or not async depending on how you look at them.
  - throw10920 2y ago
    > Again you are wrong. A forum full of people who just like to hear themselves talk. I guess it makes you feel good in some way? This is why people don't like the Rust community.

ethegwo 2y ago

the post only talks about "future state", maybe I'm not clearly to point out this. with epoll, accept syscall and future state changing is happened in the same polling, which io_uring is not. Once accept syscall is complete, future has already advanced to complete, but actually it is not at that moment in the real world Rust.

withoutboats3 2y ago
It's true, there's a necessary layer of abstraction with io-uring that doesn't exist with epoll. With epoll, the reactor just maps FDs to Wakers, and then wakes whatever Waker is waiting on that FD. Then that task does the syscall. With io-uring, instead the reactor is reading completion events from a queue. It processes those events, sets some state, and then wakes those tasks. Those tasks find the result of the syscall in that state that the reactor set. This is the difference between readiness (epoll) and completion (io-uring): with readiness the task wakes when the syscall is ready to be performed without blocking, with completion the task wakes when the syscall is already complete. When a task loses interest in an event in epoll, all that happens is it gets "spuriously awoken," so it sees there's nothing for it to do and goes back to sleep. With io-uring, the reactor needs to do more: when a task has lost interest in an incomplete event, that task needs to set the reactor into a state where instead of waking it, it will clean up the resources owned by the completion event. In the case of accept, this means closing that FD. According to your post, monoio fails to do this, and just spuriously wakes up the task, leaking the resource. The only way this relates to Rust's async model is that all futures in Rust are cancellable, so the reactor needs to handle the possibility that interest in a syscall is cancelled or the reactor is incorrect. But its completely possible to implement an io-uring reactor correctly under Rust's async model, this is just a requirement to do so.
- 0x457 2y ago
  > But its completely possible to implement an io-uring reactor correctly under Rust's async model, this is just a requirement to do so. I don't get why people say it's incompatible with rust when rust async libraries work IOCP, which follows the similar model as io-uring?