19 ms·

The State of Async Rust: Runtimes

258 points by RebootStr 3y ago

pavlov 3y ago
> "An inconvenient truth about async Rust is that libraries still need to be written against individual runtimes." In general Rust has tried hard to improve on the developer experience of C++ by providing more safety in the language and better defaults in the standard library. So it's interesting that both languages have now ended up in a similar same place for async. (C++20 coroutines finally enable sensible async libraries, but code written against a higher-level library isn't easily portable to another one even though both are using the low-level language coro primitives.) > "Freely after Stroustup: Inside Rust, there is a smaller, simpler language that is waiting to get out. It is this language that most Rust code should be written in." Maybe there's a Meta-Stroustrup's Law in effect: "Every successful language eventually becomes one which contains a smaller, simpler language struggling to get out." It happened to C++ and Java and JavaScript, now Rust seems to be reaching that point.
- Ygg2 3y ago
  In practice it's not hard to make your app support async and sync, simultaneously. Quick XML does it via macros, which looks very similar to keyword generics. Edit: > Maybe there's a Meta-Stroustrup's Law in effect: > "Every successful language eventually becomes one which contains a smaller, simpler language struggling to get out." Corollary: every simple subset of language contains missing features dearly needed by someone else.
  - 0xDEADFED5 3y ago
    quick-xml is great
  - soerxpso 3y ago
    > Corollary: every simple subset of language contains missing features dearly needed by someone else. This could even be said about larger languages, though. If Rust committed to pleasing everyone, it would have an optional garbage collector, lifetime annotations would be optional, and there would be an interpreted runtime available as an alternative to the compiler. Those are features that some people do dearly need for some tasks. There's a point where you have to stop and put limits on what the language is actually for and what it's not. It seems to me that much of the push to add async/await in the first place was from people who really should have been using Go, Java, or Node, but wanted Rust to be their "everything language." It's okay to say, "This language is for writing performant systems applications. It's not a fullstack language for writing a webapp."
davidhyde 3y ago
I’m aware that the issues are tough to work through but it’s a real shame that async traits remain in nightly. On top of this, being able to reference a set of reasonable traits from a popular library not linked to a runtime would make library writing less (runtime) siloed. For example, a library author would not have to expose their own async Read and Write traits allowing consumers of that library to use runtimes that consume those traits. The user would not then have to do the plumbing themselves.
- est31 3y ago
  async traits are in the process of being stabilized: https://github.com/rust-lang/rust/pull/115822 https://github.com/rust-lang/rust/pull/115822 Also, impl trait projections (ability to use Self::Foo associated types in async functions in traits): https://github.com/rust-lang/rust/pull/115659 https://github.com/rust-lang/rust/pull/115659
the_mitsuhiko 3y ago
> By doing so, one would set up a multi-threaded runtime which mandates that types are Send and 'static and makes it necessary to use synchronization primitives such as Arc and Mutex for all but the most trivial applications. That is a very weird argument to make. Tokio has very convenient APIs (LocalSet + spawn_local) for spawning non Send futures that you temporarily await (which really is the only useful thing non Send futures can do). If anything tokio significantly improved the user experience of async in Rust in general because it promoted Send futures.
- mre 3y ago
  There's always a trade-off. By promoting Send futures, Tokio prioritizes safety and parallelism. However, this does add complexity for developers, especially newcomers. They need to be aware of the Send and 'static requirements and might have to use synchronization primitives more often. Because of this, I think promoting Send futures as the default is the wrong way to go. LocalSet + spawn_local are great, and I wish more developers would know about them, but the Tokio tutorial [1] doesn't mention that and focuses on the multi-threaded runtime instead. AFAIK LocalSet is only mentioned in the docs [2] [1]: https://tokio.rs/tokio/tutorial https://tokio.rs/tokio/tutorial [2]: https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html
  - zozbot234 3y ago
    > LocalSet + spawn_local are great, and I wish more developers would know about them, but the Tokio tutorial doesn't mention that and focuses on the multi-threaded runtime instead. Patches welcome: https://github.com/tokio-rs/website/ https://github.com/tokio-rs/website/
  - the_mitsuhiko 3y ago
    As someone who programs async Rust since early days (where tokio did not enforce Send bounds), people build themselves into horrible patterns (myself included). Once you go deep on non sendable futures, you can quickly end up creating something you shouldn't have. So I think it's more than sensible to tell people to do the right thing and then follow up on the exceptional case via API docs or a followup guide.
dgb23 3y ago
A very informative article, that brings up important pain points and problems. I didn't agree with this sentiment though: > In a recent benchmark, async Rust was 2x faster than threads, but the absolute difference was only 10ms per request. To put this into perspective, this about as long as PHP takes to start. In other words, the difference is negligible for most applications. This statement is an ugly thorn that sticks out of the otherwise well written and reasoned article. It hurts me deep on the inside when I read stuff like this.
- Pannoniae 3y ago
  Well, one clock cycle is measured in picoseconds. That means, even if we assume 1 cycle is one nanosecond, that's 10 million CPU cycles you can do something with. That's a lot.
  - jerf 3y ago
    Your scale is off. At the moment it is still better to think of clock cycles as nanoseconds; it's still large tenths of a nanosecond. 1 cycle per nanosecond is 1GHz, so a modern processor has 2-4 cycles per nanosecond, in which it can do a lot, but not anywhere near 10 million cycles.
    - Pannoniae 3y ago
      My scale wasn't off, but I agree, it was quite misleading. When I said measured in picoseconds, I was thinking about 300-500 picoseconds roughly. However, CPUs can also execute more than one instruction in a cycle so effectively, you can have more instructions than cycles.
- kieroda 3y ago
  I agree, saying that 10ms PER REQUEST is negligible is insane. If he actually read the benchmark that was referenced, he probably didn't mean what he wrote there: the benchmark measured a 10ms difference in processing an unspecified fixed number of requests from ~100 connected clients (the benchmark article isn't actually very good, and I don't care enough to dive into the github and find out what was measured).
est31 3y ago
Regarding the one runtime point, I want to counter that it is also advantageous to not hardcode one runtime in std. This allows one to use different runtimes on webassembly. This has bitten the official async go implementation for example: https://news.ycombinator.com/item?id=37501552 https://news.ycombinator.com/item?id=37501552
- jeffparsons 3y ago
  From my occasional skimming of WebAssembly meeting minutes, I'd say that Wasm will likely grow the features required for Go to perform well. There's plenty of interest in stack switching, coroutines, etc. I work a lot more in Rust than I do in go, but I think each language made the trade-off that made most sense for that language.
  - est31 3y ago
    > I'd say that Wasm will likely grow the features required for Go Wasm has been really great at shipping the MVP, but they are pretty slow about shipping the many features that build on it. In general, this makes sense as the system can't be changed once its stable. But it also means that a lot of things are still in limbo and will probably be for the forseeable future. > I think each language made the trade-off that made most sense for that language. Definitely! Go is meant for backend application logic computing where you can provision tons of ram and ignore the issues of gc. Rust targets a larger domain, less application logic in particular but the whole range of system programming. Also including applications but also low level libraries, places without an OS, etc. I think if Rust really wants to be low-low level, then not shipping an async runtime is a must, even if the std crate is present. Providing features for libraries to support multiple runtimes? Sure. But don't apply solutions that (mostly) work for Go to Rust's problem domain.
    - jeffparsons 3y ago
      > Wasm has been really great at shipping the MVP, but they are pretty slow about shipping the many features that build on it. In general, this makes sense as the system can't be changed once its stable. But it also means that a lot of things are still in limbo and will probably be for the forseeable future. I concede that development of post-MVP development has been slow, but I am also optimistic for the near future based on recent progress. Are there particular things in limbo that you're particularly interested in or concerned about?
samsquire 3y ago
Thank you for this article. The Rust and tokio folks are working on difficult and complex problems, I appreciate and thank them for the work they're doing to improve server and desktop app performance everywhere. We all have multicore machines and it would be great if we could use more than 1/8 or 1/12 (or whatever high thread count of your beefy servers) of our hardware. The Rust multithreaded thread memory management (and Arc and so on)* causes me to be uncomfortable because of a key lesson I've learned is that you cannot scale a program by adding threads and expect it to accelerate mutate access to the SAME memory location. Single threaded memory mutation performance is a fixed known quantity. Adding threads with contention for same memory location causes throughput and latency to be slower to a particular memory location at single threaded speeds because you need mutexes or a lock free algorithm to communicate safely. To accelerate data fanout or storage (writing to memory from multiple threads) you need a shared nothing architecture or sharding. This means that when you reach for threads and I'm guessing you're wanting to reach for threads for acceleration and performance you need to design your data structures to not share memory locations. You need to shard your data. EDIT: I originally said Sync + Send.
- kzrdude 3y ago
  Send + Sync is maybe not enough but wouldn't it be possible to say: as long as the memory location is read-only you can parallelize access to it. Send + Sync helps pass the read-only data through without synchronization to all threads, while the rest of the Send and exclusive mutability system flags the tricky points for you. I can see that Send/Sync by itself does not tell you if the data is read only or just internally synchronizing mutation.
- nu11ptr 3y ago
  > We all have multicore machines and it would be great if we could use more than 1/8 or 1/12 (or whatever high thread count of your beefy servers) of our hardware. It is probably important here to realize that async solves concurrency, not parallelism. You can use async with a single threaded runtime for I/O concurrency and mix that with threads for computational parallelism for long running jobs. That said, there may be some benefit a multi-threaded runtime would have for the typical I/O bound app (after working around lifetime limitations by adding Send/Sync to data structures). This is because I/O bound programs and those requiring computation are not mutually exclusive and there is always some amount of computation going on, so there may still be some benefit. I doubt a synthetic benchmark would answer this as those typically don't measure any actual work performed, but just "requests/sec".
rob74 3y ago
Not really related to this article in particular, but I keep reading about "oxidize this" and "corrode that", which makes zero sense for a language named after a fungus (https://en.wikipedia.org/wiki/Rust_(programming_language)#Origins_(2006%E2%80%932012) https://en.wikipedia.org/wiki/Rust_(programming_language)#Or.... Ok, the proper verb for a parasitic fungus would probably be "infect", and nobody wants to go there, so I guess they just pretend that it's named after iron oxide?
- codeflo 3y ago
  Just take a look at the official Rust logo: a rusty metal gearwheel. I think they moved away from the “fungus” meaning of the name years ago.
- bmacho 3y ago
  It makes a lot of sense, since corrosion rust is a homonym for fungus rust. Also this allows a wide range of topics - related words - to work as wordplay.
- doodpants 3y ago
  Yeah, it's just like Python, which was named after Monty Python, but for some reason their logo is a pair of snakes. What does that have to do with a British comedy troupe?
- sophacles 3y ago
  The rust fungus gets it's name because it's colored like iron oxide - that is like the plants are rusting. If the fungi's name itself is a metaphor for corrosion/oxidation, why would it be improper to honor that theme?
anyfactor 3y ago
Fantastic article. My experience with Rust as an enthusiast was that tutorials tend to introduce Tokio very early on and it kinda makes Rust feel more difficult than it is. Rust's async shouldn't be taught, it should rather be discovered. The author mentions > If async is truly indispensable, consider isolating your async code from the rest of your application I think ALL async code should be generally isolated. Are there languages that provide foundational priority to asynchronous code yet supports good intermingling of sync and async in the same codebase? I maybe missing the point about isolation, but the mix of sync and async code gets bad really quick.
- mre 3y ago
  Go doesn't have native async support per se, but its approach to concurrency with goroutines and channels simplifies the process considerably. Synchronous code resembles asynchronous code, eliminating the need to isolate goroutines. Rust, on the other hand, took a different route. Green threads don't integrate smoothly with code interfacing through FFI. Moreover, Rust's async model doesn't require a garbage collector.
  - yencabulator 3y ago
    Rust's async model pretty much requires Arc, and reference counting is a simple form of a garbage collector...
- 0xDEF 3y ago
  Which Rust tutorial introduces Tokio early on?
  - anyfactor 3y ago
    With HTTP requests, you will come across reqwest and Tokio. This comment [0] introduces me to ureq and the commentor helped me to explore Rust better. I understand that making async HTTP request is a fundamental concept. However, I question why we should recommend a more complex solution when there are simpler alternatives that still leverage Rust's capabilities. [0] https://lobste.rs/s/2kvgav/learning_learn_rust#c_udauvn https://lobste.rs/s/2kvgav/learning_learn_rust#c_udauvn
K0nserv 3y ago
I agree with much of this post. I managed to avoid async Rust for three years of writing it. I do think it's the least beautiful part of Rust. My journey has been one of reaching for Arc and Mutexes and then running into problems with that approach. Relying more on channels and spawned tasks that own state i.e. Actors[0] has been a good improvement. I do think the post is a bit unfair in this sense, it rightly identifies the problems of Send and 'static. However, it also suggests Arc and Mutex are *the* solutions for shared state in async, but suggested channels for the threads example. The problem of function colouring and Send/'static bounds are the significant hurdles with async Rust, shared state is something that needs to be resolved whether using threads or async. 0: https://ryhl.io/blog/actors-with-tokio/ https://ryhl.io/blog/actors-with-tokio/
- jedisct1 3y ago
  The few projects I wrote using async Rust eventually became unmaintainable. And when things go wrong, stack traces involving Futures are impossible to understand. This is where Go really shines. Goroutines may not be "right" or "good", but they are very intuitive, and maintainable. Performance isn't bad either. In Rust, there's the May project that is very similar and should really get more attention.
  - Patrickmi 3y ago
    Here’s the problem about languages like Rust, at the very beginning of rust goals it gives you all the control while offering security and performance, want some libraries to manage some these authorities no problem, but the problem here is that if everyone want to agree one thing or feature while the language gives the programmer to full control this causes fractions in the ecosystem, 3rd party vs rust core team issues and co, a single unmaintained library can deal a massive blow to the ecosystem on like Go where “The Language makes the decision for you” it gets worst as rust isn’t a domain specific language (way more than python or java) even tho it’s a systems language this brings in different domain ideologies into the language which in turn creates massive 3rd party libraries to be able to handle those ideologies which in turn causes a massive blow to the ecosystem if more unmaintained libraries pile up. This circle will repeat its self over and over again
wokwokwok 3y ago
Really, the problem isn't tokio. The problem is this: > An inconvenient truth about async Rust is that libraries still need to be written against individual runtimes. That's really the heart of it. If it was really just a runtime, it wouldn't matter what implementation you plugged in. ...but it's not true for the rust runtime; I mean, it's understandable, how can you have one runtime that is multi-threaded and one runtime that is not, and expect to be able to seamlessly interchange them? I understand it's hard and lot of work went into this, but let's face this. This article is right: Practically speaking, tokio has become 'the' rust async runtime; but it's an opinionated runtime, that has a life cycle and direction outside of the core rust team. That wasn't where we intended to end up, and it's not a good place for things to be. I, at least, agree: avoid async. Avoid teaching rust using async. When you need to use it, partition off the async components as best you can. I <3 rust and I use it a lot, but the async story stinks. We should have an official runtime, officially managed, and guided by the same thoughts that guide the rest of the language. What we have now is a circus. After 4 years of async being in stable.
- Ygg2 3y ago
  > We should have an official runtime, officially managed, and guided by the same thoughts that guide the rest of the language. If it goes into official runtime, then backwards compatibility will kill it eventually. You'll have situation where in year 2078 someone will ask why are we still having tokio when everyone is using telepathy lib? > What we have now is a circus. After 4 years of async being in stable. It's caused by strong backwards compatibility guarantees and long RFC process + unexpected problems. Without strong backwards compatibility, no one would be using Rust. RFC exists to hash out unexpected problems but so far we can't peer in the future. Here is an example: Want to make Range from non-Copy to Copy. I.e. make a new type, rename old to new. That will be one year for RFC and two edition to stabilize circa Rust 2028. By that measure async fixes have been blazingly fast.
  - thesuperbigfrog 3y ago
    >> If it goes into official runtime, then backwards compatibility will kill it eventually. You'll have situation where in year 2078 someone will ask why are we still having tokio when everyone is using telepathy lib? This kind of situation happens and leads to a second, newer official runtime getting adopted and the older, legacy runtime being supported as long as is needed. This happened with Java's official GUT toolkits which started off with the Abstract Windowing Toolkit (AWT) and then moved to Swing and almost moved to JavaFX. Having multiple officially supported core components is not necessarily bad--it can be a sign of good backward compatibility balanced against the need to improve core components. I think it is better than the alternative: multiple unofficial, de facto standard components that are incompatible. Who knows which direction each unofficial components will go and newcomers do not know which one to choose.
Dowwie 3y ago
This article isn't covering the lack of structured concurrency and the blocking dependency on async Drop. The resulting state of async Rust includes leaks and inadequate task management. This isn't a Tokio problem but a Rust problem, and one that doesn't seem to have an answer after years of deliberation.
klysm 3y ago
I don’t write rust in any sort of large capacity, but async in rust gives me this sinking feeling that the project took a big misstep that’s going to either be permanently bad, or very painful to fix.
openasocket 3y ago
> The Original Sin of Rust async programming is making it multi-threaded by default So, obviously having to sprinkle Arc and Mutex all over the place sucks as a developer experience. But how much does that really impose in terms of runtime overhead? Both of those structures tend to perform decently in the single-threaded case. It's obviously useless work, but I'd be surprised if that shows up on any profiles as a bottleneck.
- insanitybit 3y ago
  Also important to note that while you might clone an Arc in some places (like at the beginning of a request) you can almost always just use `.as_ref()` to take a 0-cost reference to the value, thanks to borrow checking.
yellowviking 3y ago
I think rust should also forge the path where Java 21 takes (Virtual Threads) https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html https://docs.oracle.com/en/java/javase/21/core/virtual-threa...
- rwaksmunski 3y ago
  Not sure if trolling but, Rust had green threads before Rust 1.0. Like everyone sane before them they figured N:M scheduling is not the way forward and they ripped it out before going stable. That decision impressed me a lot and made me look into the language, I'm loving the journey so far.
  - hu3 3y ago
    If they are trolling then Erlang is also trolling with its legendary scheduler.
  - sapiogram 3y ago
    > Like everyone sane before them they figured N:M scheduling is not the way forward and they ripped it out before going stable. Why do you think Java recently went from N:M scheduling? Surely there's something to it?
rwaksmunski 3y ago
All I want is a basic tiny single threaded async runtime in std. No need for Send & 'static on everything. A modern single core is more than plenty for my workloads. Need more horsepower? Sure grab Tokio. I'll be fine with single async thread for IO and Rayon threadpools for heavy compute. No need to over complicate stuff.
eximius 3y ago
One of the common comments in this thread is "can't we just make standard interfaces in std?" "well, no, sync+send is hard" I can't help but wonder if there are two sets of interfaces necessary? a set of standard single-threaded traits and a set of multi-threaded traits? would that be sufficient? As an aside, what workloads require true multithreaded reactors as opposed to a runtime which uses multiple singlethreaded reactors?
- yencabulator 3y ago
  > As an aside, what workloads require true multithreaded reactors as opposed to a runtime which uses multiple singlethreaded reactors? For example, DataFusion spreads query processing over multiple cores by having the data flow be a "streaming DAG of record batches" (or something like that), as in futures::Stream. https://docs.rs/datafusion/latest/datafusion/ https://docs.rs/datafusion/latest/datafusion/ https://docs.rs/datafusion/latest/datafusion/execution/trait.RecordBatchStream.html https://docs.rs/datafusion/latest/datafusion/execution/trait...
moomin 3y ago
So I know _nothing_ about Rust, but I know Tasks in C# and one of the most important concepts is that a variable can be “async local” i.e. local to the async “thread”. So I wonder if the problem is that rust doesn’t have a lifetime specific to an “async thread”. As long as it’s well understood that the variables don’t leave that context, I think everything is easier to reason about.
- insanitybit 3y ago
  You can move a variable into a task, or even borrow it across the task, and things work perfectly fine. Frankly, this article is blowing things massively out of proportion.
  - moomin 3y ago
    As I say, I really don’t understand and this probably isn’t the place to educate me, but this isn’t the only article I’ve seen that regards Send + ‘static as a) mandatory and b) problematic.
    - insanitybit 3y ago
      For sure, some people definitely think this is problematic. But I wonder how much of that is "this is problematic to my philosophy of programming" versus "this actually slows me down when writing code".
insanitybit 3y ago
I had already replied to this article over on lobste.rs but I'll link to that from here. https://lobste.rs/s/iovz9o/state_async_rust https://lobste.rs/s/iovz9o/state_async_rust The tl;dr is that I think this entire async concern stuff is ridiculously overblown. I suspect the vast majority of Rust devs, like myself, not only accept the current state as "fine" (couple things to work on) but are very happy with the decisions tokio made. Things like "oh no you have to use Arc" are really acting like that's more than a trivial change. Or "you have to use Mutex" when you don't. Or "you can accidentally block" as if that's not the case with regular threads too, and in that case I acknowledge that async can make things trickier there. Or "tokio is so popular" as if that's not exactly because of the decisions it made early on that appealed to rust developers. Sorry but it's just not that big of a deal. The warts I run into are in cases where I'm trying to do stuff like zero copy, async, abstracted deserialization. That can be a pain right now (and is being worked on). 99% of the time it's a matter of just writing `async` or `await` and not worrying about anything. In fact, I almost never use `tokio::spawn` anyways except at the binary level - these problems virtually do not impact me. Source: I have written 10s of thousands of lines of async Rust, probably 100s of thousands.
- linza 3y ago
  > these problems virtually do not impact me. I'm confused, what are you suggesting? That Rust hits a global sweet spot already, the complainers are struggling because they are holding it wrong, and there shouldn't be an attempt to change anything?
  - insanitybit 3y ago
    I think I was clear - that the critiques are overblown and that these problems aren't nearly as significant as portrayed. Sure, there may be some "holding it wrong" going on, idk. A couple of blog posts aren't really representative of the overall feeling from devs like myself - that things are more or less fine. As for changing things, I wouldn't really change any issues brought up here. I'd like to see some things smoothed out, like async traits, tooling to identify hot loops that are blocking without yielding, and things like that. Otherwise, nope, working as intended as far as I'm concerned.
Zuiii 3y ago
An async runtime should have been a core part of the language from day one. Yes, make it user replaceable if you want (like the allocator) but one must be provided by the standard library. No buts.

258 points by RebootStr 3y ago

> "An inconvenient truth about async Rust is that libraries still need to be written against individual runtimes." In general Rust has tried hard to improve on the developer experience of C++ by providing more safety in the language and better defaults in the standard library. So it's interesting that both languages have now ended up in a similar same place for async. (C++20 coroutines finally enable sensible async libraries, but code written against a higher-level library isn't easily portable to another one even though both are using the low-level language coro primitives.) > "Freely after Stroustup: Inside Rust, there is a smaller, simpler language that is waiting to get out. It is this language that most Rust code should be written in." Maybe there's a Meta-Stroustrup's Law in effect: "Every successful language eventually becomes one which contains a smaller, simpler language struggling to get out." It happened to C++ and Java and JavaScript, now Rust seems to be reaching that point.

Ygg2 3y ago
In practice it's not hard to make your app support async and sync, simultaneously. Quick XML does it via macros, which looks very similar to keyword generics. Edit: > Maybe there's a Meta-Stroustrup's Law in effect: > "Every successful language eventually becomes one which contains a smaller, simpler language struggling to get out." Corollary: every simple subset of language contains missing features dearly needed by someone else.
- 0xDEADFED5 3y ago
  quick-xml is great
- soerxpso 3y ago
  > Corollary: every simple subset of language contains missing features dearly needed by someone else. This could even be said about larger languages, though. If Rust committed to pleasing everyone, it would have an optional garbage collector, lifetime annotations would be optional, and there would be an interpreted runtime available as an alternative to the compiler. Those are features that some people do dearly need for some tasks. There's a point where you have to stop and put limits on what the language is actually for and what it's not. It seems to me that much of the push to add async/await in the first place was from people who really should have been using Go, Java, or Node, but wanted Rust to be their "everything language." It's okay to say, "This language is for writing performant systems applications. It's not a fullstack language for writing a webapp."

davidhyde 3y ago

I’m aware that the issues are tough to work through but it’s a real shame that async traits remain in nightly. On top of this, being able to reference a set of reasonable traits from a popular library not linked to a runtime would make library writing less (runtime) siloed. For example, a library author would not have to expose their own async Read and Write traits allowing consumers of that library to use runtimes that consume those traits. The user would not then have to do the plumbing themselves.

est31 3y ago
async traits are in the process of being stabilized: https://github.com/rust-lang/rust/pull/115822 https://github.com/rust-lang/rust/pull/115822 Also, impl trait projections (ability to use Self::Foo associated types in async functions in traits): https://github.com/rust-lang/rust/pull/115659 https://github.com/rust-lang/rust/pull/115659

the_mitsuhiko 3y ago

> By doing so, one would set up a multi-threaded runtime which mandates that types are Send and 'static and makes it necessary to use synchronization primitives such as Arc and Mutex for all but the most trivial applications. That is a very weird argument to make. Tokio has very convenient APIs (LocalSet + spawn_local) for spawning non Send futures that you temporarily await (which really is the only useful thing non Send futures can do). If anything tokio significantly improved the user experience of async in Rust in general because it promoted Send futures.

mre 3y ago
There's always a trade-off. By promoting Send futures, Tokio prioritizes safety and parallelism. However, this does add complexity for developers, especially newcomers. They need to be aware of the Send and 'static requirements and might have to use synchronization primitives more often. Because of this, I think promoting Send futures as the default is the wrong way to go. LocalSet + spawn_local are great, and I wish more developers would know about them, but the Tokio tutorial [1] doesn't mention that and focuses on the multi-threaded runtime instead. AFAIK LocalSet is only mentioned in the docs [2] [1]: https://tokio.rs/tokio/tutorial https://tokio.rs/tokio/tutorial [2]: https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html
- zozbot234 3y ago
  > LocalSet + spawn_local are great, and I wish more developers would know about them, but the Tokio tutorial doesn't mention that and focuses on the multi-threaded runtime instead. Patches welcome: https://github.com/tokio-rs/website/ https://github.com/tokio-rs/website/
- the_mitsuhiko 3y ago
  As someone who programs async Rust since early days (where tokio did not enforce Send bounds), people build themselves into horrible patterns (myself included). Once you go deep on non sendable futures, you can quickly end up creating something you shouldn't have. So I think it's more than sensible to tell people to do the right thing and then follow up on the exceptional case via API docs or a followup guide.

dgb23 3y ago

A very informative article, that brings up important pain points and problems. I didn't agree with this sentiment though: > In a recent benchmark, async Rust was 2x faster than threads, but the absolute difference was only 10ms per request. To put this into perspective, this about as long as PHP takes to start. In other words, the difference is negligible for most applications. This statement is an ugly thorn that sticks out of the otherwise well written and reasoned article. It hurts me deep on the inside when I read stuff like this.

Pannoniae 3y ago
Well, one clock cycle is measured in picoseconds. That means, even if we assume 1 cycle is one nanosecond, that's 10 million CPU cycles you can do something with. That's a lot.
- jerf 3y ago
  Your scale is off. At the moment it is still better to think of clock cycles as nanoseconds; it's still large tenths of a nanosecond. 1 cycle per nanosecond is 1GHz, so a modern processor has 2-4 cycles per nanosecond, in which it can do a lot, but not anywhere near 10 million cycles.
  - Pannoniae 3y ago
    My scale wasn't off, but I agree, it was quite misleading. When I said measured in picoseconds, I was thinking about 300-500 picoseconds roughly. However, CPUs can also execute more than one instruction in a cycle so effectively, you can have more instructions than cycles.
kieroda 3y ago
I agree, saying that 10ms PER REQUEST is negligible is insane. If he actually read the benchmark that was referenced, he probably didn't mean what he wrote there: the benchmark measured a 10ms difference in processing an unspecified fixed number of requests from ~100 connected clients (the benchmark article isn't actually very good, and I don't care enough to dive into the github and find out what was measured).

est31 3y ago

Regarding the one runtime point, I want to counter that it is also advantageous to not hardcode one runtime in std. This allows one to use different runtimes on webassembly. This has bitten the official async go implementation for example: https://news.ycombinator.com/item?id=37501552 https://news.ycombinator.com/item?id=37501552

jeffparsons 3y ago
From my occasional skimming of WebAssembly meeting minutes, I'd say that Wasm will likely grow the features required for Go to perform well. There's plenty of interest in stack switching, coroutines, etc. I work a lot more in Rust than I do in go, but I think each language made the trade-off that made most sense for that language.
- est31 3y ago
  > I'd say that Wasm will likely grow the features required for Go Wasm has been really great at shipping the MVP, but they are pretty slow about shipping the many features that build on it. In general, this makes sense as the system can't be changed once its stable. But it also means that a lot of things are still in limbo and will probably be for the forseeable future. > I think each language made the trade-off that made most sense for that language. Definitely! Go is meant for backend application logic computing where you can provision tons of ram and ignore the issues of gc. Rust targets a larger domain, less application logic in particular but the whole range of system programming. Also including applications but also low level libraries, places without an OS, etc. I think if Rust really wants to be low-low level, then not shipping an async runtime is a must, even if the std crate is present. Providing features for libraries to support multiple runtimes? Sure. But don't apply solutions that (mostly) work for Go to Rust's problem domain.
  - jeffparsons 3y ago
    > Wasm has been really great at shipping the MVP, but they are pretty slow about shipping the many features that build on it. In general, this makes sense as the system can't be changed once its stable. But it also means that a lot of things are still in limbo and will probably be for the forseeable future. I concede that development of post-MVP development has been slow, but I am also optimistic for the near future based on recent progress. Are there particular things in limbo that you're particularly interested in or concerned about?

samsquire 3y ago

Thank you for this article. The Rust and tokio folks are working on difficult and complex problems, I appreciate and thank them for the work they're doing to improve server and desktop app performance everywhere. We all have multicore machines and it would be great if we could use more than 1/8 or 1/12 (or whatever high thread count of your beefy servers) of our hardware. The Rust multithreaded thread memory management (and Arc and so on)* causes me to be uncomfortable because of a key lesson I've learned is that you cannot scale a program by adding threads and expect it to accelerate mutate access to the SAME memory location. Single threaded memory mutation performance is a fixed known quantity. Adding threads with contention for same memory location causes throughput and latency to be slower to a particular memory location at single threaded speeds because you need mutexes or a lock free algorithm to communicate safely. To accelerate data fanout or storage (writing to memory from multiple threads) you need a shared nothing architecture or sharding. This means that when you reach for threads and I'm guessing you're wanting to reach for threads for acceleration and performance you need to design your data structures to not share memory locations. You need to shard your data. EDIT: I originally said Sync + Send.

kzrdude 3y ago
Send + Sync is maybe not enough but wouldn't it be possible to say: as long as the memory location is read-only you can parallelize access to it. Send + Sync helps pass the read-only data through without synchronization to all threads, while the rest of the Send and exclusive mutability system flags the tricky points for you. I can see that Send/Sync by itself does not tell you if the data is read only or just internally synchronizing mutation.
nu11ptr 3y ago
> We all have multicore machines and it would be great if we could use more than 1/8 or 1/12 (or whatever high thread count of your beefy servers) of our hardware. It is probably important here to realize that async solves concurrency, not parallelism. You can use async with a single threaded runtime for I/O concurrency and mix that with threads for computational parallelism for long running jobs. That said, there may be some benefit a multi-threaded runtime would have for the typical I/O bound app (after working around lifetime limitations by adding Send/Sync to data structures). This is because I/O bound programs and those requiring computation are not mutually exclusive and there is always some amount of computation going on, so there may still be some benefit. I doubt a synthetic benchmark would answer this as those typically don't measure any actual work performed, but just "requests/sec".

rob74 3y ago

Not really related to this article in particular, but I keep reading about "oxidize this" and "corrode that", which makes zero sense for a language named after a fungus (https://en.wikipedia.org/wiki/Rust_(programming_language)#Origins_(2006%E2%80%932012) https://en.wikipedia.org/wiki/Rust_(programming_language)#Or.... Ok, the proper verb for a parasitic fungus would probably be "infect", and nobody wants to go there, so I guess they just pretend that it's named after iron oxide?

codeflo 3y ago
Just take a look at the official Rust logo: a rusty metal gearwheel. I think they moved away from the “fungus” meaning of the name years ago.
bmacho 3y ago
It makes a lot of sense, since corrosion rust is a homonym for fungus rust. Also this allows a wide range of topics - related words - to work as wordplay.
doodpants 3y ago
Yeah, it's just like Python, which was named after Monty Python, but for some reason their logo is a pair of snakes. What does that have to do with a British comedy troupe?
sophacles 3y ago
The rust fungus gets it's name because it's colored like iron oxide - that is like the plants are rusting. If the fungi's name itself is a metaphor for corrosion/oxidation, why would it be improper to honor that theme?

anyfactor 3y ago

Fantastic article. My experience with Rust as an enthusiast was that tutorials tend to introduce Tokio very early on and it kinda makes Rust feel more difficult than it is. Rust's async shouldn't be taught, it should rather be discovered. The author mentions > If async is truly indispensable, consider isolating your async code from the rest of your application I think ALL async code should be generally isolated. Are there languages that provide foundational priority to asynchronous code yet supports good intermingling of sync and async in the same codebase? I maybe missing the point about isolation, but the mix of sync and async code gets bad really quick.

mre 3y ago
Go doesn't have native async support per se, but its approach to concurrency with goroutines and channels simplifies the process considerably. Synchronous code resembles asynchronous code, eliminating the need to isolate goroutines. Rust, on the other hand, took a different route. Green threads don't integrate smoothly with code interfacing through FFI. Moreover, Rust's async model doesn't require a garbage collector.
- yencabulator 3y ago
  Rust's async model pretty much requires Arc, and reference counting is a simple form of a garbage collector...
0xDEF 3y ago
Which Rust tutorial introduces Tokio early on?
- anyfactor 3y ago
  With HTTP requests, you will come across reqwest and Tokio. This comment [0] introduces me to ureq and the commentor helped me to explore Rust better. I understand that making async HTTP request is a fundamental concept. However, I question why we should recommend a more complex solution when there are simpler alternatives that still leverage Rust's capabilities. [0] https://lobste.rs/s/2kvgav/learning_learn_rust#c_udauvn https://lobste.rs/s/2kvgav/learning_learn_rust#c_udauvn

K0nserv 3y ago

I agree with much of this post. I managed to avoid async Rust for three years of writing it. I do think it's the least beautiful part of Rust. My journey has been one of reaching for Arc and Mutexes and then running into problems with that approach. Relying more on channels and spawned tasks that own state i.e. Actors[0] has been a good improvement. I do think the post is a bit unfair in this sense, it rightly identifies the problems of Send and 'static. However, it also suggests Arc and Mutex are *the* solutions for shared state in async, but suggested channels for the threads example. The problem of function colouring and Send/'static bounds are the significant hurdles with async Rust, shared state is something that needs to be resolved whether using threads or async. 0: https://ryhl.io/blog/actors-with-tokio/ https://ryhl.io/blog/actors-with-tokio/

jedisct1 3y ago
The few projects I wrote using async Rust eventually became unmaintainable. And when things go wrong, stack traces involving Futures are impossible to understand. This is where Go really shines. Goroutines may not be "right" or "good", but they are very intuitive, and maintainable. Performance isn't bad either. In Rust, there's the May project that is very similar and should really get more attention.
- Patrickmi 3y ago
  Here’s the problem about languages like Rust, at the very beginning of rust goals it gives you all the control while offering security and performance, want some libraries to manage some these authorities no problem, but the problem here is that if everyone want to agree one thing or feature while the language gives the programmer to full control this causes fractions in the ecosystem, 3rd party vs rust core team issues and co, a single unmaintained library can deal a massive blow to the ecosystem on like Go where “The Language makes the decision for you” it gets worst as rust isn’t a domain specific language (way more than python or java) even tho it’s a systems language this brings in different domain ideologies into the language which in turn creates massive 3rd party libraries to be able to handle those ideologies which in turn causes a massive blow to the ecosystem if more unmaintained libraries pile up. This circle will repeat its self over and over again

wokwokwok 3y ago

Really, the problem isn't tokio. The problem is this: > An inconvenient truth about async Rust is that libraries still need to be written against individual runtimes. That's really the heart of it. If it was really just a runtime, it wouldn't matter what implementation you plugged in. ...but it's not true for the rust runtime; I mean, it's understandable, how can you have one runtime that is multi-threaded and one runtime that is not, and expect to be able to seamlessly interchange them? I understand it's hard and lot of work went into this, but let's face this. This article is right: Practically speaking, tokio has become 'the' rust async runtime; but it's an opinionated runtime, that has a life cycle and direction outside of the core rust team. That wasn't where we intended to end up, and it's not a good place for things to be. I, at least, agree: avoid async. Avoid teaching rust using async. When you need to use it, partition off the async components as best you can. I <3 rust and I use it a lot, but the async story stinks. We should have an official runtime, officially managed, and guided by the same thoughts that guide the rest of the language. What we have now is a circus. After 4 years of async being in stable.

Ygg2 3y ago
> We should have an official runtime, officially managed, and guided by the same thoughts that guide the rest of the language. If it goes into official runtime, then backwards compatibility will kill it eventually. You'll have situation where in year 2078 someone will ask why are we still having tokio when everyone is using telepathy lib? > What we have now is a circus. After 4 years of async being in stable. It's caused by strong backwards compatibility guarantees and long RFC process + unexpected problems. Without strong backwards compatibility, no one would be using Rust. RFC exists to hash out unexpected problems but so far we can't peer in the future. Here is an example: Want to make Range from non-Copy to Copy. I.e. make a new type, rename old to new. That will be one year for RFC and two edition to stabilize circa Rust 2028. By that measure async fixes have been blazingly fast.
- thesuperbigfrog 3y ago
  >> If it goes into official runtime, then backwards compatibility will kill it eventually. You'll have situation where in year 2078 someone will ask why are we still having tokio when everyone is using telepathy lib? This kind of situation happens and leads to a second, newer official runtime getting adopted and the older, legacy runtime being supported as long as is needed. This happened with Java's official GUT toolkits which started off with the Abstract Windowing Toolkit (AWT) and then moved to Swing and almost moved to JavaFX. Having multiple officially supported core components is not necessarily bad--it can be a sign of good backward compatibility balanced against the need to improve core components. I think it is better than the alternative: multiple unofficial, de facto standard components that are incompatible. Who knows which direction each unofficial components will go and newcomers do not know which one to choose.

Dowwie 3y ago

This article isn't covering the lack of structured concurrency and the blocking dependency on async Drop. The resulting state of async Rust includes leaks and inadequate task management. This isn't a Tokio problem but a Rust problem, and one that doesn't seem to have an answer after years of deliberation.

klysm 3y ago

I don’t write rust in any sort of large capacity, but async in rust gives me this sinking feeling that the project took a big misstep that’s going to either be permanently bad, or very painful to fix.

openasocket 3y ago

> The Original Sin of Rust async programming is making it multi-threaded by default So, obviously having to sprinkle Arc and Mutex all over the place sucks as a developer experience. But how much does that really impose in terms of runtime overhead? Both of those structures tend to perform decently in the single-threaded case. It's obviously useless work, but I'd be surprised if that shows up on any profiles as a bottleneck.

insanitybit 3y ago
Also important to note that while you might clone an Arc in some places (like at the beginning of a request) you can almost always just use `.as_ref()` to take a 0-cost reference to the value, thanks to borrow checking.

yellowviking 3y ago

I think rust should also forge the path where Java 21 takes (Virtual Threads) https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html https://docs.oracle.com/en/java/javase/21/core/virtual-threa...

rwaksmunski 3y ago
Not sure if trolling but, Rust had green threads before Rust 1.0. Like everyone sane before them they figured N:M scheduling is not the way forward and they ripped it out before going stable. That decision impressed me a lot and made me look into the language, I'm loving the journey so far.
- hu3 3y ago
  If they are trolling then Erlang is also trolling with its legendary scheduler.
- sapiogram 3y ago
  > Like everyone sane before them they figured N:M scheduling is not the way forward and they ripped it out before going stable. Why do you think Java recently went from N:M scheduling? Surely there's something to it?

rwaksmunski 3y ago

All I want is a basic tiny single threaded async runtime in std. No need for Send & 'static on everything. A modern single core is more than plenty for my workloads. Need more horsepower? Sure grab Tokio. I'll be fine with single async thread for IO and Rayon threadpools for heavy compute. No need to over complicate stuff.

eximius 3y ago

One of the common comments in this thread is "can't we just make standard interfaces in std?" "well, no, sync+send is hard" I can't help but wonder if there are two sets of interfaces necessary? a set of standard single-threaded traits and a set of multi-threaded traits? would that be sufficient? As an aside, what workloads require true multithreaded reactors as opposed to a runtime which uses multiple singlethreaded reactors?

yencabulator 3y ago
> As an aside, what workloads require true multithreaded reactors as opposed to a runtime which uses multiple singlethreaded reactors? For example, DataFusion spreads query processing over multiple cores by having the data flow be a "streaming DAG of record batches" (or something like that), as in futures::Stream. https://docs.rs/datafusion/latest/datafusion/ https://docs.rs/datafusion/latest/datafusion/ https://docs.rs/datafusion/latest/datafusion/execution/trait.RecordBatchStream.html https://docs.rs/datafusion/latest/datafusion/execution/trait...

moomin 3y ago

So I know _nothing_ about Rust, but I know Tasks in C# and one of the most important concepts is that a variable can be “async local” i.e. local to the async “thread”. So I wonder if the problem is that rust doesn’t have a lifetime specific to an “async thread”. As long as it’s well understood that the variables don’t leave that context, I think everything is easier to reason about.

insanitybit 3y ago
You can move a variable into a task, or even borrow it across the task, and things work perfectly fine. Frankly, this article is blowing things massively out of proportion.
- moomin 3y ago
  As I say, I really don’t understand and this probably isn’t the place to educate me, but this isn’t the only article I’ve seen that regards Send + ‘static as a) mandatory and b) problematic.
  - insanitybit 3y ago
    For sure, some people definitely think this is problematic. But I wonder how much of that is "this is problematic to my philosophy of programming" versus "this actually slows me down when writing code".

insanitybit 3y ago

I had already replied to this article over on lobste.rs but I'll link to that from here. https://lobste.rs/s/iovz9o/state_async_rust https://lobste.rs/s/iovz9o/state_async_rust The tl;dr is that I think this entire async concern stuff is ridiculously overblown. I suspect the vast majority of Rust devs, like myself, not only accept the current state as "fine" (couple things to work on) but are very happy with the decisions tokio made. Things like "oh no you have to use Arc" are really acting like that's more than a trivial change. Or "you have to use Mutex" when you don't. Or "you can accidentally block" as if that's not the case with regular threads too, and in that case I acknowledge that async can make things trickier there. Or "tokio is so popular" as if that's not exactly because of the decisions it made early on that appealed to rust developers. Sorry but it's just not that big of a deal. The warts I run into are in cases where I'm trying to do stuff like zero copy, async, abstracted deserialization. That can be a pain right now (and is being worked on). 99% of the time it's a matter of just writing `async` or `await` and not worrying about anything. In fact, I almost never use `tokio::spawn` anyways except at the binary level - these problems virtually do not impact me. Source: I have written 10s of thousands of lines of async Rust, probably 100s of thousands.

linza 3y ago
> these problems virtually do not impact me. I'm confused, what are you suggesting? That Rust hits a global sweet spot already, the complainers are struggling because they are holding it wrong, and there shouldn't be an attempt to change anything?
- insanitybit 3y ago
  I think I was clear - that the critiques are overblown and that these problems aren't nearly as significant as portrayed. Sure, there may be some "holding it wrong" going on, idk. A couple of blog posts aren't really representative of the overall feeling from devs like myself - that things are more or less fine. As for changing things, I wouldn't really change any issues brought up here. I'd like to see some things smoothed out, like async traits, tooling to identify hot loops that are blocking without yielding, and things like that. Otherwise, nope, working as intended as far as I'm concerned.

Zuiii 3y ago

An async runtime should have been a core part of the language from day one. Yes, make it user replaceable if you want (like the allocator) but one must be provided by the standard library. No buts.