5 ms·

Rust to stabilize `async fn` and return-position `impl Trait` in traits

231 points by saghm 3y ago

vlovich123 3y ago
Very important milestone reached for async programming, not to mention that this is actually a primitive that just happens to be useful for async as well. It’s not always needed but when you find yourself needing it, it becomes a major point of friction. Congrats to the team! Excited for dynamic dispatch and capture rules to be coming soon.
- ForkMeOnTinder 3y ago
  It's useful even without async too. Our codebase has some trait methods that return boxed iterators and I'm very much looking forward to switching them to impl Iterator instead.
adamch 3y ago
Nice job! I'm glad the team was able to ship a useful subset of the desired functionality. Yes the current implementation shipping next week has limitations, but they're clearly marked, the compiler helpfully explains them and warns you away from some sharp edges. I'm glad the Rust project is willing to ship useful but limited features quickly, see how people use them, and then iterate and slowly remove the restrictions in the future. I think it'll be more productive than taking another 3 years to solve all the remaining rough edges and problems.
- tempaccount420 3y ago
  In general, Rust should commit more to stabilizing important features like this. I think because of how easy it is to switch to nightly, there isn't as much pressure to stabilize features as there should be.
andy_xor_andrew 3y ago
I'm confused about the example for trait_variant: ``` #[trait_variant::make(HttpService: Send)] ... ``` In the given example, they create a trait with an async fn that returns HtmlBody, then the trait_variant macro makes a duplicate version of the trait but additionally with a Send bound on the future. Makes sense, because users commonly need the future to be Send. But... what's the point of having two traits? Could there not just be one trait, with the Send bound? Doesn't the Send bound only increase the places it can be used? Are there scenarios where you need the future to not be Send?
- dureuill 3y ago
  It increases the places it can be used, but it decreases what type can be returned. Not all types can be sent between threads (in particular, types that feature unsynchronized mutability through shared references are not Send). If you executor is local to one thread, it can be a blocker if a trait you use suddenly requires your future to be Send.
  - andy_xor_andrew 3y ago
    ah, ok. so this is for scenarios where you, the library author, define the trait, and users will implement it. But since different users have different requirements, you provide two+ trait definitions, so they can pick if they need Send or not.
    - Arnavion 3y ago
      Yes. Say the HttpService trait comes from some third-party library. I implement that trait on my server, and then I run my server on a single-threaded executor. (This is analogous to how Iterator is in libstd but I can implement it on my own type and then use that Iterator with a for-loop / combinator in my own code.) My implementation returns Futures that are not Send and my executor does not need them to be Send, so it would be wasteful if the trait required the Futures to be Send.
andy_xor_andrew 3y ago
this is so cool :) one other question for async Rust experts... is it trivial to replace every use of the #[async_trait] with this, once it releases? async_trait was just a macro for adding a bunch of `Box<Pin<dyn Future<Output = BlahBlah> + OtherJunk>` and so on and so on. Is async_trait entirely obsolete now, or are there still situations where async_trait must be used, where builtin async will not? edit: if I read one more paragraph I would have seen the answer, it's right there in the post :)
- vlovich123 3y ago
  For anyone looking for a TLDR, dynamic dispatch is the major missing piece. The only other reason is if you need to support a compiler version < 1.75
  - pitaj 3y ago
    I've seen dynamic dispatch mentioned twice. What do you mean? Isn't that just `fn foo() -> Box<dyn Future<...>>`?
    - gdcbe 3y ago
      Dynamic dispatch is where you call a method on a Box<dyn T> type, where T is a trait. Cfr: https://doc.rust-lang.org/1.8.0/book/trait-objects.html https://doc.rust-lang.org/1.8.0/book/trait-objects.html This in contrast to generics which is static.
  - gdcbe 3y ago
    Yeah and I was trying to implement their workaround for missing dynamic dispatch in a generic-heavy multithreaded async (tokio, hyper) context.., Cfr: https://github.com/plabayo/tower-async/pull/12 https://github.com/plabayo/tower-async/pull/12 So far didn’t manage to produce a box that works without everything else in my tower async stack having to need send sync trait bounds as well… and of course needing nightly ‘call(): Send’ return bounds. Despite that, haven’t found a reasonable working solution to support that in ‘tower-async’ Might be just me still having to b learn a lot. More then happy to hear feedbacks add help on these matters. And of course this is a WIP. So fully understand. Respect to the entire async wg team. Onwards and forward.
monocasa 3y ago
What's the timeline for allowing async fn with dynamic dispatch?
- duped 3y ago
  You can do trait Trait { fn f(&self) -> futures::BoxFuture<...> { let fut = async move { /* ... */ }; fut.boxed() } } (which is essentially what `#[async_trait]` desugars to) I'm not sure how you would even support dynamic dispatch in the general case without implicitly boxing the return value, which would be deeply unpopular within the Rust community and ecosystem. If we're saying we're ok with implicit allocations for futures then the whole "zero cost abstraction" of async/await seems pointless.
  - pitaj 3y ago
    I guess the difference is that that isn't an async function. It's a normal function that returns a boxed future.
    - duped 3y ago
      An async function in Rust is a function that returns a future.
  - bbatha 3y ago
    Note that an "implicit allocation" doesn't need to be a heap allocation, it just needs to be dynamically sized. Stabilizing the allocator api (which seems tantalizingly close) and someway to pass a non-default allocator to the implicit box should cover zero cost sensitive folks. Also note that "zero cost abstraction" means that "zero cost" over implementing it by hand there is no way to return an owned dyn sized type in current rust. Syntax that returns a box is inherently zero cost because there's no other way to spell it. C++ `co_await` also forces an allocation for similar reasons and lets the optimizer get rid of them in the static case which predictably is a mixed bag in practice; at least in rust you can guarantee the static sized version doesn't allocate. That said I don't like forcing an implicit allocation it feels against Rust's ethos and the language will likely have to have better handling of dyn types writ large to get there.
gdcbe 3y ago
Congratulations to all involved. Huge milestone! That said, I’ve been using it seriously on nightly for a while already… and must say its ergonomics are off putting. For example I maintain https://github.com/plabayo/tower-async https://github.com/plabayo/tower-async (a fork of tower), and on its own it looks to work (except for the boxing and dynamic dispatching others have already discussed). But once you throw it into the tokio ecosystem, building on top of something like hyper, and you suddenly are back into nightly territory due to having to specify trait bound for your async fn trait methods (eg: call(): Send). It works, and you can see in my early WIP proxy repo (https://github.com/plabayo/rama https://github.com/plabayo/rama). It’s not pretty but does work… that said it does put you in a nightly position once again due to having to specify Send/Sync trait bounds of trait async fn methods opaque futures… In contrast the RFC that would allow me to write ‘type Future = impl Future<…’ seems to fit a lot better in the existing tokio ecosystem. Just my 2 cents. Anyway, I get its hard work and WIP, so congratulations again and thx for all the work! (Edit: I do understand that the main source of my issues is due to the combination of writing very generic code and using multithreaded async (via tokio). Still, it’s not pretty. But perhaps it’s also because I still have a lot to learn on how to use and write this. The latter is my hope)
- kurtbuilds 3y ago
  Can you elaborate on why the `mixing` approach doesn't work? Specifically, if you want send bounds, doing this: trait HttpService: Send { fn fetch(&self, url: Url) -> impl Future<Output = HtmlBody> + Send; } impl HttpService for MyService { async fn fetch(&self, url: Url) -> HtmlBody { // This works, as long as `do_fetch(): Send`! self.client.do_fetch(url).await.into_body() } }
  - gdcbe 3y ago
    Of course that works. But only as long as everything is always hardcoded as Send. Which is a known limitation, so no complaints. Edit: also the problem is about the method async returned future trait bounds, not the Self.
- aliceryhl 3y ago
cchance 3y ago
impl trait returns is so cool
- gdcbe 3y ago
  Amen to that. Love it sooo much. Looking forward to the day I can use that for associated types. If I can get that for my birthday in May, it would be magical.
qaq 3y ago
Congrats! Looks like it was a monumental effort
Zuiii 3y ago
Does Rust still require you to rely on external dependencies (or roll out your own async runtime) to run async code?
- steveklabnik 3y ago
  Yes, and that is unlikely to ever change.
  - vlovich123 3y ago
    But it would be nice to standardize the IO / time interfaces so that swapping out runtimes doesn't require writing a custom adapter layer :). I know you know, just saying that's really probably what OP meant.
  - Zuiii 3y ago
    That's unfortunate. I hope this gets fixed.
    - jrpelkonen 3y ago
      Cargo makes it rather seamless to include the async runtime that best suits your needs. I am curious why do you think this is a major issue?
leshow 3y ago
nice! been waiting on this one for ages. Any idea if removing the implicit box-ing that async_trait uses has a measurable impact on performance? Are there some cases, say with really big futures, where removing the boxing actually hurts performance?
- tmandry 3y ago
  I don't have numbers handy, but I can say with confidence that the answer to both questions is "yes". It depends on the use case. Boxing is a useful tool and in many cases the overhead is minimal, but you don't want to use it all the time. Likewise, static inlining can bite you in deeply nested cases and I've heard of this happening too. The future we're working toward is for this to be a decision you can make locally without introducing compatibility hazards or having to change a bunch of code. Ideally, one day you can even ask the compiler to alert you to potential performance hazards or make reasonable default choices for you...
deleted 3y ago
[deleted]

231 points by saghm 3y ago

Very important milestone reached for async programming, not to mention that this is actually a primitive that just happens to be useful for async as well. It’s not always needed but when you find yourself needing it, it becomes a major point of friction. Congrats to the team! Excited for dynamic dispatch and capture rules to be coming soon.

ForkMeOnTinder 3y ago
It's useful even without async too. Our codebase has some trait methods that return boxed iterators and I'm very much looking forward to switching them to impl Iterator instead.

adamch 3y ago

Nice job! I'm glad the team was able to ship a useful subset of the desired functionality. Yes the current implementation shipping next week has limitations, but they're clearly marked, the compiler helpfully explains them and warns you away from some sharp edges. I'm glad the Rust project is willing to ship useful but limited features quickly, see how people use them, and then iterate and slowly remove the restrictions in the future. I think it'll be more productive than taking another 3 years to solve all the remaining rough edges and problems.

tempaccount420 3y ago
In general, Rust should commit more to stabilizing important features like this. I think because of how easy it is to switch to nightly, there isn't as much pressure to stabilize features as there should be.

andy_xor_andrew 3y ago

I'm confused about the example for trait_variant: ``` #[trait_variant::make(HttpService: Send)] ... ``` In the given example, they create a trait with an async fn that returns HtmlBody, then the trait_variant macro makes a duplicate version of the trait but additionally with a Send bound on the future. Makes sense, because users commonly need the future to be Send. But... what's the point of having two traits? Could there not just be one trait, with the Send bound? Doesn't the Send bound only increase the places it can be used? Are there scenarios where you need the future to not be Send?

dureuill 3y ago
It increases the places it can be used, but it decreases what type can be returned. Not all types can be sent between threads (in particular, types that feature unsynchronized mutability through shared references are not Send). If you executor is local to one thread, it can be a blocker if a trait you use suddenly requires your future to be Send.
- andy_xor_andrew 3y ago
  ah, ok. so this is for scenarios where you, the library author, define the trait, and users will implement it. But since different users have different requirements, you provide two+ trait definitions, so they can pick if they need Send or not.
  - Arnavion 3y ago
    Yes. Say the HttpService trait comes from some third-party library. I implement that trait on my server, and then I run my server on a single-threaded executor. (This is analogous to how Iterator is in libstd but I can implement it on my own type and then use that Iterator with a for-loop / combinator in my own code.) My implementation returns Futures that are not Send and my executor does not need them to be Send, so it would be wasteful if the trait required the Futures to be Send.

andy_xor_andrew 3y ago

this is so cool :) one other question for async Rust experts... is it trivial to replace every use of the #[async_trait] with this, once it releases? async_trait was just a macro for adding a bunch of `Box<Pin<dyn Future<Output = BlahBlah> + OtherJunk>` and so on and so on. Is async_trait entirely obsolete now, or are there still situations where async_trait must be used, where builtin async will not? edit: if I read one more paragraph I would have seen the answer, it's right there in the post :)

vlovich123 3y ago
For anyone looking for a TLDR, dynamic dispatch is the major missing piece. The only other reason is if you need to support a compiler version < 1.75
- pitaj 3y ago
  I've seen dynamic dispatch mentioned twice. What do you mean? Isn't that just `fn foo() -> Box<dyn Future<...>>`?
  - gdcbe 3y ago
    Dynamic dispatch is where you call a method on a Box<dyn T> type, where T is a trait. Cfr: https://doc.rust-lang.org/1.8.0/book/trait-objects.html https://doc.rust-lang.org/1.8.0/book/trait-objects.html This in contrast to generics which is static.
- gdcbe 3y ago
  Yeah and I was trying to implement their workaround for missing dynamic dispatch in a generic-heavy multithreaded async (tokio, hyper) context.., Cfr: https://github.com/plabayo/tower-async/pull/12 https://github.com/plabayo/tower-async/pull/12 So far didn’t manage to produce a box that works without everything else in my tower async stack having to need send sync trait bounds as well… and of course needing nightly ‘call(): Send’ return bounds. Despite that, haven’t found a reasonable working solution to support that in ‘tower-async’ Might be just me still having to b learn a lot. More then happy to hear feedbacks add help on these matters. And of course this is a WIP. So fully understand. Respect to the entire async wg team. Onwards and forward.

monocasa 3y ago

What's the timeline for allowing async fn with dynamic dispatch?

duped 3y ago
You can do trait Trait { fn f(&self) -> futures::BoxFuture<...> { let fut = async move { /* ... */ }; fut.boxed() } } (which is essentially what `#[async_trait]` desugars to) I'm not sure how you would even support dynamic dispatch in the general case without implicitly boxing the return value, which would be deeply unpopular within the Rust community and ecosystem. If we're saying we're ok with implicit allocations for futures then the whole "zero cost abstraction" of async/await seems pointless.
- pitaj 3y ago
  I guess the difference is that that isn't an async function. It's a normal function that returns a boxed future.
  - duped 3y ago
    An async function in Rust is a function that returns a future.
- bbatha 3y ago
  Note that an "implicit allocation" doesn't need to be a heap allocation, it just needs to be dynamically sized. Stabilizing the allocator api (which seems tantalizingly close) and someway to pass a non-default allocator to the implicit box should cover zero cost sensitive folks. Also note that "zero cost abstraction" means that "zero cost" over implementing it by hand there is no way to return an owned dyn sized type in current rust. Syntax that returns a box is inherently zero cost because there's no other way to spell it. C++ `co_await` also forces an allocation for similar reasons and lets the optimizer get rid of them in the static case which predictably is a mixed bag in practice; at least in rust you can guarantee the static sized version doesn't allocate. That said I don't like forcing an implicit allocation it feels against Rust's ethos and the language will likely have to have better handling of dyn types writ large to get there.

gdcbe 3y ago

Congratulations to all involved. Huge milestone! That said, I’ve been using it seriously on nightly for a while already… and must say its ergonomics are off putting. For example I maintain https://github.com/plabayo/tower-async https://github.com/plabayo/tower-async (a fork of tower), and on its own it looks to work (except for the boxing and dynamic dispatching others have already discussed). But once you throw it into the tokio ecosystem, building on top of something like hyper, and you suddenly are back into nightly territory due to having to specify trait bound for your async fn trait methods (eg: call(): Send). It works, and you can see in my early WIP proxy repo (https://github.com/plabayo/rama https://github.com/plabayo/rama). It’s not pretty but does work… that said it does put you in a nightly position once again due to having to specify Send/Sync trait bounds of trait async fn methods opaque futures… In contrast the RFC that would allow me to write ‘type Future = impl Future<…’ seems to fit a lot better in the existing tokio ecosystem. Just my 2 cents. Anyway, I get its hard work and WIP, so congratulations again and thx for all the work! (Edit: I do understand that the main source of my issues is due to the combination of writing very generic code and using multithreaded async (via tokio). Still, it’s not pretty. But perhaps it’s also because I still have a lot to learn on how to use and write this. The latter is my hope)

kurtbuilds 3y ago
Can you elaborate on why the `mixing` approach doesn't work? Specifically, if you want send bounds, doing this: trait HttpService: Send { fn fetch(&self, url: Url) -> impl Future<Output = HtmlBody> + Send; } impl HttpService for MyService { async fn fetch(&self, url: Url) -> HtmlBody { // This works, as long as `do_fetch(): Send`! self.client.do_fetch(url).await.into_body() } }
- gdcbe 3y ago
  Of course that works. But only as long as everything is always hardcoded as Send. Which is a known limitation, so no complaints. Edit: also the problem is about the method async returned future trait bounds, not the Self.
aliceryhl 3y ago

cchance 3y ago

impl trait returns is so cool

gdcbe 3y ago
Amen to that. Love it sooo much. Looking forward to the day I can use that for associated types. If I can get that for my birthday in May, it would be magical.

qaq 3y ago

Congrats! Looks like it was a monumental effort

Zuiii 3y ago

Does Rust still require you to rely on external dependencies (or roll out your own async runtime) to run async code?

steveklabnik 3y ago
Yes, and that is unlikely to ever change.
- vlovich123 3y ago
  But it would be nice to standardize the IO / time interfaces so that swapping out runtimes doesn't require writing a custom adapter layer :). I know you know, just saying that's really probably what OP meant.
- Zuiii 3y ago
  That's unfortunate. I hope this gets fixed.
  - jrpelkonen 3y ago
    Cargo makes it rather seamless to include the async runtime that best suits your needs. I am curious why do you think this is a major issue?

leshow 3y ago

nice! been waiting on this one for ages. Any idea if removing the implicit box-ing that async_trait uses has a measurable impact on performance? Are there some cases, say with really big futures, where removing the boxing actually hurts performance?

tmandry 3y ago
I don't have numbers handy, but I can say with confidence that the answer to both questions is "yes". It depends on the use case. Boxing is a useful tool and in many cases the overhead is minimal, but you don't want to use it all the time. Likewise, static inlining can bite you in deeply nested cases and I've heard of this happening too. The future we're working toward is for this to be a decision you can make locally without introducing compatibility hazards or having to change a bunch of code. Ideally, one day you can even ask the compiler to alert you to potential performance hazards or make reasonable default choices for you...

deleted 3y ago

[deleted]