13 ms·
How Rust optimizes async/await
- Paul-ish 7y agoI think the year is off by 1 in the blog post, or the title needs a 2018 tag.
- tmandry 7y agoThat's what I get for publishing late at night. Fixed, thanks!
- giancarlostoro 7y agoFormer, or that's some impressive time travel knowledge.
- orthecreedence 7y agoRustradamus.
- weiming 7y agoAs a newcomer to Rust, wishing that this post was one of the first ones I've read about this topic. It took scouring through many many posts, some of them here on HN, to be able to grasp some of the same idea. (I may not be alone, judging from the very long discussion the other day: https://news.ycombinator.com/item?id=20719095 https://news.ycombinator.com/item?id=20719095)
- hathawsh 7y agoDevelopment of high quality async support in Rust is happening right now, so remember to wear a hard hat. ;-) I like to watch https://areweasyncyet.rs/ https://areweasyncyet.rs/ and https://this-week-in-rust.org/ https://this-week-in-rust.org/ to see where things are.
- steveklabnik 7y agoPart of this is just that the design has been in the works for four years and has changed significantly during that time; it’s only now that things are almost stable that it’s worthwhile to write these kinds of things.
- GolDDranks 7y agoWell, it's got still three months until it lands stable, maybe it's just that the time hasn't been ripe for great, understandable posts about the feature until recently.
- strictfp 7y agoWe're getting generators as well? Awesome.
- weiming 7y agoA future is a one-shot generator, give or take. Good reading list at the bottom of https://areweasyncyet.rs https://areweasyncyet.rs, starting with this post that uses generators as an example: https://boats.gitlab.io/blog/post/2018-01-25-async-i-self-referential-structs/ https://boats.gitlab.io/blog/post/2018-01-25-async-i-self-re...
- tmandry 7y agoNot necessarily. They're an implementation detail of the compiler, and aren't fully baked yet to boot. But there's plenty of reason to want generators, including the fact that they let you build streams. And the fact that async/await relies heavily on them has pushed the implementation much closer to being ready. I hope we get them at some point!
- richardwhiuk 7y agoThey are available in nightly - https://doc.rust-lang.org/beta/unstable-book/language-features/generators.html https://doc.rust-lang.org/beta/unstable-book/language-featur... - but are very unstable. I think "at some point" is the best answer :)
- steveklabnik 7y agoTo be clear on how unstable: they have not received an actual RFC yet, so they’re barely designed at all. It will be some time before they’re stable, as in some sense, they haven’t even started the process. I’d imagine that they would go through the stages faster than many other features, though, given that they implementation will have received a lot of testing via async/await.
- continuational 7y agoI don't quite follow. What exactly is the overhead that other languages have for futures that is eliminated here?
- tmandry 7y agoMost languages allocate every future (and sub-future, and sub-sub-future) separately on the heap. This leads to some overhead, allocating and deallocating space to store our task state. In Rust, you can "inline" an entire chain of futures into a single heap allocation.
- pjmlp 7y agoIn .NET something similar is possible via ValueTask.
- Matthias247 7y agoValueTask is more of an optimization for scenarios where the Future (or Task<T>) is often intended to be ready immediately (synchronously). For those cases its wasteful to first allocate a continuation on the heap, and then not to use it - because the code after the await can directly be executed. The introduction of ValueTask allowed C# code to only perform dynamic allocations whenever the task definitely needed to be yielded - opposed to one allocation for every Task<T>. However it doesn't allow for guaranteed avoidance of allocations - like Rusts model can do. However on the positive side removing the allocations for those cases is probably good enough since the other yields are of low-frequency (when waiting for IO). And Rust code currently requires allocations like Box<dyn Future> in order to make certain things work (like type-erasure and dynamically dispatched interfaces) that are not necessarily required in .NET in the non-yielded case. From an ergonomic perspective I definitely prefer .NETs easy-by-default model which is still highly optimizable up to the point of avoiding all allocations. But I understand why this wouldn't have worked for Rust and it's constraints (no GC, no classes, no runtime).
- pjmlp 7y ago
- pcwalton 7y agoAs a reminder, you don't need to use async/await to implement socket servers in Rust. You can use threads, and they scale quite well. M:N scheduling was found to be slower than 1:1 scheduling on Linux, which is why Rust doesn't use that solution. Async/await is a great feature for those who require performance beyond what threads (backed by either 1:1 or M:N implementations) can provide. One of the major reasons behind the superior performance of async/await futures relative to threads/goroutines/etc. is that async/await compiles to a state machine in the manner described in this post, so a stack is not needed.
- jnordwick 7y agoepoll io loop performs better for most network io though and is simplier to manage when you have to start dealing with out of band issues (like efficient hearbeats - every time I've had a conversation without how to move some of the heartbeat code over to async it comes down to just accepting it isn't going to be as efficient as my c++ implementation and either strain heavily of accept over publication). Last time I saw there was still a couple extra allocations going on too in the compiler (I was told they were being worked on) and basically the default executor, tokio, wasn't very efficient at changing events in the queue (requiring a an extra self signal to get the job done). I'd be interesting to see how little cost these are, because there is defintely a cost to the generator implementation. Yes, if I wrong a generator to do this, I couldn't write it better, but I wouldn't write a generator (and that would be a very odd definition of zero-cost there anything can be called zero cost even GC as long as it is implemented well - well, that depends on if rustc saves unnecessary state). > Additionally, it should allow miri to validate the unsafe code inside our generators, checking for uses of unsafe that result in undefined behavior. This is really useful as a lot of the buffer handing code needs to use unsafe for efficienty issues. And the enums sharing values is nice too - hopefully the extra pointer derferences can be optimized out. I do worry though about all this state sitting on the head and ruining cache locality on function calls though.
- rapsey 7y agoHonestly I don’t get it why simply using mio (epoll, kqueue, iocp wrapper) is so unpopular.
- MuffinFlavored 7y agoHow does `yield` work under the hood? Does it add a reference to some array, with code to loop over all the references with some small timeout until the reference status changes from "pending" to "completed"?
- tmandry 7y agoGenerators do nothing unless you call their resume() method. resume moves the generator from the last state it was in to the next yield (or return). Internally, when the code hits a yield, it's happening inside the resume method. yield works by saving the current state of the generator in the object (see e.g. resume_from in the post), and returning from resume().
- TheCoelacanth 7y agoIt gets converted into a state machine. || { yield 2; yield 3; yield 5; } will get converted to a struct that implements the Generator trait[1] with a resume method something like fn resume(self: Pin<&mut Self>) -> GeneratorState<i32, ()> { match self.next_state { 0 => { self.next_state = 1; Yielded(2) }, 1 => { self.next_state = 2; Yielded(3) }, 2 => { self.next_state = 3; Yielded(5) }, _ => Complete(()) } } Local variables become fields in the struct and if you have more complex control flow, the state could end up jumping around instead of just increasing by one each time. It's nothing that you couldn't write by hand, but it would be very tedious to do so. [1] https://doc.rust-lang.org/beta/unstable-book/language-features/generators.html https://doc.rust-lang.org/beta/unstable-book/language-featur...
- kzrdude 7y agoJust like for async, borrow across yield points here are a special feature that you couldn't implement in Rust as an open coded state machine, you'd have to find a workaround.
- zackmorris 7y agoThis is one of the most concise tutorials on how generators, coroutines and futures/promises are related (from first principles) that I've seen. I'm hopeful that eventually promises and async/await fade into history as a fad that turned out to be too unwieldy. I think that lightweight processes with no shared memory, connected by streams (the Erlang/Elixer, Go and Actor model) are the way to go. The advantage to using async/await seems to be to avoid the dynamic stack allocation of coroutines, which can probably be optimized away anyway. So I don't see a strong enough advantage in moving from blocking to nonblocking code. Or to rephrase, I don't see the advantage in moving from deterministic to nondeterministic code. I know that all of the edge cases in a promise chain can be handled, but I have yet to see it done well in deployed code. Which makes me think that it's probably untenable for the mainstream. So I'd vote to add generators to the Rust spec in order to make coroutines possible, before I'd add futures/promises and async/await. But maybe they are all equivalent, so if we have one, we can make all of them, not sure.
- jnwatson 7y agoRegarding determinism, async is way more deterministic than multiple threads, because you don’t have arbitrary point where execution contexts can change.
- zackmorris 7y agoThat's true in a way, but only for multithreaded code. Multi-process code with full isolation uses different metaphors like joining threads within higher order functions to achieve parallelism in code that looks single-threaded. For example, lisp-based languages like Clojure can be statically analyzed and then parallelized so that all noninteracting code runs in its own process. This can also be done for code that operates on vectors like MATLAB and TensorFlow. For me, isolated processes under the Actor model in languages like Elixer/Erlang and Go is much simpler conceptually than async/await, which is only one step above promises/futures, which is only one step above callback hell. I know that the web world uses async/await for now, but someday I think that will be replaced with something that works more like Go.
- tmandry 7y ago
- saurik 7y agoDoes anyone know how Rust's implementation compares to C++2a's? The C++ people seem to have spent a lot of time creating an extremely generic framework for async/await wherein it is easy to change out how the scheduler works (I currently have a trivial work stack, but am going to be moving to something more akin to a deadline scheduler in the near future for a large codebase I am working with, which needs to be able to associate extra prioritization data into the task object, something that is reasonably simple to do with await_transform). I am also under the impression that existing implementation in LLVM already does some of these optimizations that Rust says they will get around to doing (as the C++ people also care a lot about zero-cost).
- tmandry 7y agoDisclaimer: I'm not an expert on the proposal, but have looked at it some, and can offer my impressions here. (Sorry, this got a bit long!) The C++ proposal definitely attacks the problem from a different angle than Rust. One somewhat surface-level difference is that it implements co_yield in terms of co_await, which is the opposite of Rust implementing await in terms of yield. Another difference is that in Rust, all heap allocations of your generators/futures are explicit. In C++, technically every initialization of a sub-coroutine starts defaults to being a new heap allocation. I don't want to spread FUD: my understanding is that the vast majority of these are optimized out by the compiler. But one downside of this approach is that you could change your code and accidentally disable one of these optimizations. In Rust, all the "state inlining" is explicitly done as part of the language. This means that in cases where you can't inline state, you must introduce an explicit indirection. (Imagine, say, a recursive generator - it's impossible to inline inside of itself! When you recurse, you must allocate the new generator on the heap, inside a Box.) To be clear, the optimizations I'm talking about in the blog post are all implemented today. I'll be covering what they do and don't do, as well as future work needed, in future blog posts. One benefit of C++ that you allude to is that there are a lot of extension points. I admit to not fully understanding what each one of them is for, but my feeling is that some of it comes from approaching the problem differently. Some of it absolutely represents missing features in Rust's initial implementation. But as I say in the post, we can and will add more features on a rolling basis. The way I would approach the specific problem you mention is with a custom executor. When you write the executor, you control how new tasks are scheduled, and can add an API that allows specifying a task priority. You can also allow modifying this priority within the task: when you poll a task, set a thread-local variable to point to that task. Then inside the task, you can gain a reference to yourself and modify your priority.
- fnord77 7y agoI'm a big user of Rust, but I'm kinda dismayed that async IO wasn't part of the original design. It's nice they're making Futures a zero-cost abstraction, but it feels like it is at the expense of ergonomics.
- tmandry 7y agoFutures were developed outside Rust core, in a third-party library, before being brought into the language. Working with them in combinator form definitely was less ergonomic, but async/await fixes that.
- steveklabnik 7y agoFun fact: there was a future type in the rust standard library, long, long ago. https://doc.rust-lang.org/0.10/sync/struct.Future.html https://doc.rust-lang.org/0.10/sync/struct.Future.html
- brson 7y agoThat version of Rust also did async I/O in the runtime. Async I/O has always been part of Rust. The model changed because there was too much overhead doing it the more ergonomic way and it got booted out of the runtime.
- steveklabnik 7y agoYep, this is a great point. Someday, we should get a book about the history of Rust together...
- tmandry 7y agoI didn’t know about this.. I’d love to read that book :)
- tracker1 7y agoYou have to prioritize something. Development of other features was prioritized over async first. Futures support came later, and async later still. It's an iterative approach. In the end, it let people get a LOT of value out of rust well before async was fully baked. For myself, I'd been waiting for the syntax to finalize as I'm still learning and didn't want to really delve into old and new ways, though I'm sure I will see them in practice. For others, depending on your needs, if you didn't need it, or willing to jump through the hoops, you could still get value along the way.
- emmanueloga_ 7y agoRelated questions: does anybody know: 1) which was the first language that introduced the async/await keywords? I want to say C#, but I’m not sure. 2) are there any papers that describe the code transformation to state machine that is commonly performed to implement these keywords?
- TkTech 7y agoFor #1, there's a well-sourced Stack Overflow post https://softwareengineering.stackexchange.com/a/377514 https://softwareengineering.stackexchange.com/a/377514.
- emmanueloga_ 7y agoCool, I'll take a look! I was quickly searching around and found this paper [1]: "Pause 'n' play: formalizing asynchronous C#". It looks promising, although it is behind a paywall ;-(. Also, the keyword "formalizing" tells me that maybe this goes a bit deeper than the kind of description I'm looking for... 1: https://dl.acm.org/citation.cfm?id=2367181 https://dl.acm.org/citation.cfm?id=2367181
- girvo 7y agosci-hub.tw usually helps with getting papers that are behind paywalls!
- deleted 7y ago[deleted]
- Walther 7y agoThank you for the great writeup <3 Eagerly waiting for the upcoming parts - please keep it up!