6 ms·

Great article! Love these types of deep dives into optimizations. Hope the project goal works out! I've felt before that compilers often don't put much effort

by hmry 5mo ago

Great article! Love these types of deep dives into optimizations. Hope the project goal works out! I've felt before that compilers often don't put much effort into optimizing the "trivial" cases. Overly dramatic title for the content, though. I would have clicked "Async Rust Optimizations the Compiler Still Misses" too you know

Animats 5mo ago
Agree on title. Too dramatic. The author seems to be obsessing about the overhead for trivial functions. He's bothered by overhead for states for "panicked" and "returned". That's not a big problem. Most useful async blocks are big enough that the overhead for the error cases disappears. He may have a point about lack of inlining. But what tends to limit capacity for large numbers of activities is the state space required per activity.
- molf 5mo ago
  > Most useful async blocks are big enough that the overhead for the error cases disappears. Is it really though? In my experience many Rust applications/libraries can be quite heavy on the indirection. One of the points from the article is that contrary to sync Rust, in async Rust each indirection has a runtime cost. Example from the article: async fn bar(blah: SomeType) -> OtherType { foo(blah).await } I would naively expect the above to be a 'free' indirection, paying only a compile-time cost for the compiler to inline the code. But after reading the article I understand this is not true, and it has a runtime cost as well.
- swiftcoder 5mo ago
  > Most useful async blocks are big enough that the overhead for the error cases disappears. Most useful async blocks are deeply nested, so the overhead compounds rapidly. Check the size of futures in a decently large Tokio codebase sometime
- dathinab 5mo ago
  > Agree on title. Too dramatic. not just too dramatic given that all the things they list are non essential optimizations, and some fall under "micro optimizations I wouldn't be sure rust even wants", and given how far the current async is away from it's old MVP state, it's more like outright dishonest then overly dramatic like the kind of click bait which is saying the author does cares neither about respecting the reader nor cares about honest communication, which for someone wanting to do open source contributions is kinda ... not so clever through in general I agree rust should have more HIR/MIR optimizations, at least in release mode. E.g. its very common that a async function is not pub and in all places directly awaited (or other wise can be proven to only be called once), in that case neither `Returned` nor `Panicked` is needed, as it can't be called again after either. Similar `Unresumed` is not needed either as you can directly call the code up to the first await (and with such a transform their points about "inlining" and "asyncfns without await still having a state machine" would also "just go away"TM, at least in some places.). Similar the whole `.map_or(a,b)` family of functions is IMHO a anti-pattern, introducing more function with unclear operator ordering and removal of the signaling `unwrap_` and no benefits outside of minimal shortening a `.map(b).unwrap_or(a)` and some micro opt. is ... not productive on a already complicated language. Instead guaranteed optimizations for the kind of patterns a `.map(b).unwrap_or(a)` inline to would be much better.
- algesten 5mo ago
  He's optimizing for embedded no-std situation. These things do matter in constrained environments.
  - Animats 5mo ago
    He briefly mentions microcontrollers, but doesn't go into details. That might make sense, for microcontrollers which implement some kind of protocol talking to something.
- repelsteeltje 5mo ago
  > [...] That's not a big problem [...] Depends somewhat on your expectations, I suppose. Compared to Python, Java, sure, but Rust off course strives to offer "zero-cost" high level concepts. I think the critique is in the same realm of C++'s std::function. Convenience, sure, but far from zero-cost.
  - pjmlp 5mo ago
    To the point it got replaced by std::function_ref() in C++26.
    - repelsteeltje 5mo ago
      Exactly. And I guess that is also the gist of the article: async Rust needs additional TLC.
- selfhoster1312 5mo ago
  In my experience, it's not uncommon to have an async trait method for which many implementations are actually synchronous. For example, different tables in your DB need to perform some calculations, but only some tables reference other tables. In that case, the method needs to be async and take a handle to the DB as parameter, but many table entries can perform the calculation on their own without using the handle (or any async operation). This may look like a case of over-optimization, but given how many times i've seen this pattern, i assume it builds up to a lot of unnecessary fluff in huge codebases. To be clear, in that case, the concern is not really about runtime speed (which is super fast), but rather about code bloat for compilation time and binary size.
diondokter 5mo ago
So on the title, I picked this because it's simply the truth. Since async landed in 2019 or so, not much has changed. Yes, we can have async in traits and closures now. But those are updates to the typesystem, not to the async machinery itself. Wakers are a little bit easier to work with, but that's an update to std/core. As I understand it, the people who landed async Rust were quite burnt out and got less active and no one has picked up the torch. (Though there's 1 PR open from some google folk that will optimize how captured variables are laid out in memory, which is really nice to have) Since I and the people I work with are heavy async users, I think it's maybe up to me to do it or at least start it. Free as in puppy I guess. So yeah, the title is a little baitey, but I do stand behind it.
- mplanchard 5mo ago
  Some of the burnout no doubt being due to the catastrophizing of every decision by the community and the extreme rhetoric used across the board. Great to see people wanting to get involved with the project, though. That’s the beauty of open source: if it aggravates you, you can fix it.
  - selfhoster1312 5mo ago
    As an example of this, i remember a huge debate at the time about `await foo()` vs `foo().await` syntax. The community was really divided on that one, and there was a lot of drama because that's the kind of design decision you can't really walk back from. Retrospectively, i think everyone is satisfied with the adopted syntax.
    - jurgenburgen 5mo ago
      > Retrospectively, i think everyone is satisfied with the adopted syntax. Maybe it’s a case of agree and commit, since it can’t really be walked back.
      - diondokter 5mo ago
        Various prominent people have said years after that .await was the correct choice after all

by hmry 5mo ago

Animats 5mo ago

Agree on title. Too dramatic. The author seems to be obsessing about the overhead for trivial functions. He's bothered by overhead for states for "panicked" and "returned". That's not a big problem. Most useful async blocks are big enough that the overhead for the error cases disappears. He may have a point about lack of inlining. But what tends to limit capacity for large numbers of activities is the state space required per activity.

molf 5mo ago
> Most useful async blocks are big enough that the overhead for the error cases disappears. Is it really though? In my experience many Rust applications/libraries can be quite heavy on the indirection. One of the points from the article is that contrary to sync Rust, in async Rust each indirection has a runtime cost. Example from the article: async fn bar(blah: SomeType) -> OtherType { foo(blah).await } I would naively expect the above to be a 'free' indirection, paying only a compile-time cost for the compiler to inline the code. But after reading the article I understand this is not true, and it has a runtime cost as well.
swiftcoder 5mo ago
> Most useful async blocks are big enough that the overhead for the error cases disappears. Most useful async blocks are deeply nested, so the overhead compounds rapidly. Check the size of futures in a decently large Tokio codebase sometime
dathinab 5mo ago
> Agree on title. Too dramatic. not just too dramatic given that all the things they list are non essential optimizations, and some fall under "micro optimizations I wouldn't be sure rust even wants", and given how far the current async is away from it's old MVP state, it's more like outright dishonest then overly dramatic like the kind of click bait which is saying the author does cares neither about respecting the reader nor cares about honest communication, which for someone wanting to do open source contributions is kinda ... not so clever through in general I agree rust should have more HIR/MIR optimizations, at least in release mode. E.g. its very common that a async function is not pub and in all places directly awaited (or other wise can be proven to only be called once), in that case neither `Returned` nor `Panicked` is needed, as it can't be called again after either. Similar `Unresumed` is not needed either as you can directly call the code up to the first await (and with such a transform their points about "inlining" and "asyncfns without await still having a state machine" would also "just go away"TM, at least in some places.). Similar the whole `.map_or(a,b)` family of functions is IMHO a anti-pattern, introducing more function with unclear operator ordering and removal of the signaling `unwrap_` and no benefits outside of minimal shortening a `.map(b).unwrap_or(a)` and some micro opt. is ... not productive on a already complicated language. Instead guaranteed optimizations for the kind of patterns a `.map(b).unwrap_or(a)` inline to would be much better.
algesten 5mo ago
He's optimizing for embedded no-std situation. These things do matter in constrained environments.
- Animats 5mo ago
  He briefly mentions microcontrollers, but doesn't go into details. That might make sense, for microcontrollers which implement some kind of protocol talking to something.
repelsteeltje 5mo ago
> [...] That's not a big problem [...] Depends somewhat on your expectations, I suppose. Compared to Python, Java, sure, but Rust off course strives to offer "zero-cost" high level concepts. I think the critique is in the same realm of C++'s std::function. Convenience, sure, but far from zero-cost.
- pjmlp 5mo ago
  To the point it got replaced by std::function_ref() in C++26.
  - repelsteeltje 5mo ago
    Exactly. And I guess that is also the gist of the article: async Rust needs additional TLC.
selfhoster1312 5mo ago
In my experience, it's not uncommon to have an async trait method for which many implementations are actually synchronous. For example, different tables in your DB need to perform some calculations, but only some tables reference other tables. In that case, the method needs to be async and take a handle to the DB as parameter, but many table entries can perform the calculation on their own without using the handle (or any async operation). This may look like a case of over-optimization, but given how many times i've seen this pattern, i assume it builds up to a lot of unnecessary fluff in huge codebases. To be clear, in that case, the concern is not really about runtime speed (which is super fast), but rather about code bloat for compilation time and binary size.

diondokter 5mo ago

So on the title, I picked this because it's simply the truth. Since async landed in 2019 or so, not much has changed. Yes, we can have async in traits and closures now. But those are updates to the typesystem, not to the async machinery itself. Wakers are a little bit easier to work with, but that's an update to std/core. As I understand it, the people who landed async Rust were quite burnt out and got less active and no one has picked up the torch. (Though there's 1 PR open from some google folk that will optimize how captured variables are laid out in memory, which is really nice to have) Since I and the people I work with are heavy async users, I think it's maybe up to me to do it or at least start it. Free as in puppy I guess. So yeah, the title is a little baitey, but I do stand behind it.

mplanchard 5mo ago
Some of the burnout no doubt being due to the catastrophizing of every decision by the community and the extreme rhetoric used across the board. Great to see people wanting to get involved with the project, though. That’s the beauty of open source: if it aggravates you, you can fix it.
- selfhoster1312 5mo ago
  As an example of this, i remember a huge debate at the time about `await foo()` vs `foo().await` syntax. The community was really divided on that one, and there was a lot of drama because that's the kind of design decision you can't really walk back from. Retrospectively, i think everyone is satisfied with the adopted syntax.
  - jurgenburgen 5mo ago
    > Retrospectively, i think everyone is satisfied with the adopted syntax. Maybe it’s a case of agree and commit, since it can’t really be walked back.
    - diondokter 5mo ago
      Various prominent people have said years after that .await was the correct choice after all