10 ms·

Why Async Rust?

212 points by withoutboats3 3y ago

hdevalence 3y ago
A lot of discussion about async Rust assumes that the reason one would want to use async/Futures is for performance and scalability reasons. Personally, though, I would strongly prefer to use async rather than explicit threading even for cases where performance wasn’t the highest priority. The conceptual model is just better. Futures allow you to cleanly express composition of sub-tasks in a way that explicit threading doesn’t: https://monkey.org/~marius/futures-arent-ersatz-threads.html https://monkey.org/~marius/futures-arent-ersatz-threads.html
- riku_iki 3y ago
  Futures as a concept are orthogonal to async, they can totally work in explicit threading model.
  - _ahxg 3y ago
    Do you know an example of such case in the Rust ecosystem?
    - riku_iki 3y ago
      No, I don't know, I am talking about general concept. In Java, Future.get() blocks current thread, and it is trivially integrated into explicit threading programming. In Rust, Future.poll() is not blocking, and one would need to rely on some async framework, or build own event loop which can potentially block thread.
    - tux3 3y ago
      It's worth noting that a thread's JoinHandle provides a similar interface. You can spawn your tasks, store the JoinHandle "futures", and wait for completion whenever you need the result. A difference being that Futures do nothing until polled, while threads start on their own, but that's arguably a helpful simplification for this purpose.
      - convolvatron 3y ago
        this I really found disconcerting - the model I often want is I want to start some work, and then join at some later point - or even chain directly into the next task. but instead I start a future, and then to run it at all I need to wait for the result. I understand the there are tools to effect this, but it really leaves you wondering - what did I just do? start an async task and then .. block on it in order to get it to execute?
        laurencerowe 3y ago
        I'm not an expert, but my understanding is that Rust makes a distinction between Futures and Tasks, the top level Futures that are run by the executor. In JavaScript terms Futures are more like sugar around callbacks, they don't do anything until you call/poll them. Tasks are independent entities like Promises which are being run by the executor, though they may currently be blocked on other tasks. > the model I often want is I want to start some work, and then join at some later point - or even chain directly into the next task. Rust wants you to do this the other way around. First chain together your futures so that when you start the top level one as a task there is a single state machine for it to run.
  - jupp0r 3y ago
    They can be used that way, but you end up with exactly the same problems that async programming aims to avoid (performance, deadlocks, your business logic being cluttered with low level implementation details).
    - riku_iki 3y ago
      > that async programming aims to avoid (performance, deadlocks, your business logic being cluttered with low level implementation details). I disagree with you, my code looks safe and simple with explicit blocking threading, and at the same time is much simpler to reason about what is going on and tune in contrast to async frameworks which hide most of the details under the hood. You can argue about performance, that async/epoll/etc allows to avoid spawning thousands of threads and remove some overhead, but there is no much benchmarks in internet (per my research) which would say that this performance overhead is large.
      - jupp0r 3y ago
        If you are using explicit blocking, share data between threads and have not run into deadlocks then your application is trivial (which is great if it solves your problem).
        riku_iki 3y ago
        Could you explain how sharing data between threads is different in async programming and blocking programming?
        jupp0r 3y ago
        You can minimize sharing data between threads because it's easier to have data affinity with threads (ie only thread A will read or write to a piece of data). You can still access that data from multiple modules because the whole thread is never blocked waiting for IO (because of async). An extreme example is nodejs, where you only have one thread, can concurrently do thousands of things and never have to coordinate (ie via mutexes) data access.
- ori_b 3y ago
  Futures are nicely equivalent to one shot channels with threads.
  - littlestymaar 3y ago
    With a nice syntax sugar on top, but yeah pretty much.
  - pornel 3y ago
    Not quite: Rust futures also have immediate cancellation and easy timeouts that can be imposed externally on any future. In threads that perform blocking I/O you don't get that, and need to support timeouts and cancellation explicitly in every blocking call.
Animats 3y ago
He gives the business case: It was clear, though usually left unsaid, that what Rust needed to succeed was industry adoption, so that it could continue to receive support once Mozilla stopped being willing to fund an experimental new language. And it was clear that the most likely path to short-term industry adoption was in network services, especially those with a performance profile that compelled them at the time to be written in C/C++ ... The other advantage of network services was that this wing of the software industry has the flexibility and appetite to rapidly adopt a new technology like Rust. The other domains were - and are! - viable long term opportunities for Rust, but they were seen as not as quick to adopt new technology (embedded), depended on a new platform that had not yet seen widespread adoption itself (WebAssembly), or were not a particularly lucrative industrial application that could lead to funding for the language (CLIs). I drove at async/await with the diligent fervor of the assumption that Rust’s survival depended on this feature. ... Many of the most prominent sponsors of the Rust Foundation, especially those who pay developers, depend on async/await to write high performance network services in Rust as one of their primary use cases that justify their funding. Using async/await for embedded systems or kernel programming is also a growing area of interest with a bright future. Async/await has been so successful that the most common complaint about it is that the ecosystem is too centered on it, rather than “normal” Rust. I don’t know what to tell users who would rather just use threads and blocking IO. Certainly, I think there are a lot of systems for which that is a reasonable approach. And nothing in the Rust language prevents them from doing it. Their objection seems to be that the ecosystem on crates.io, especially for writing network services, is centered on using async/await. ... None of us can control what everyone else decides to work on, and the fact of the matter is just that most people who release networking-related libraries on crates.io want to use async Rust, whether for business reasons or just out of interest. I’d like it to be easier to use those libraries in a non-async context (e.g. by bringing a pollster-like API into the standard library), but it’s hard to know what to say to people who’s gripe is that the people putting code online for free don’t have exactly the same use case as them. Well, that says it. Rust has pivoted to web stuff. For which Go is probably better suited. Good-bye, Rust as a systems language, or for game development. Younger programmers will probably still be seeing C/C++ buffer overflows in 2050. The technical problem is that pure async, like JavaScript, is fine, and pure threading, like classic Rust, is fine. But the combination is awful.
IshKebab 3y ago
> Context-switching between the kernel and userspace is expensive in terms of CPU cycles. > OS threads have a large pre-allocated stack, which increases per-thread memory overhead. It feels like it would be totally possibly to improve OSes to remove this limitation. Is anyone actually working on that? For example I don't see why you couldn't have growable stacks for threads. Or have first class hardware support for context switching. (Yes that would take a long time to arrive.)
- zeroCalories 3y ago
  > It feels like it would be totally possibly to improve OSes to remove this limitation. Is anyone actually working on that? This is just my guess, but I assume they don't use growable stacks for the same reason Rust doesn't. C doesn't have a garbage collector, and making a fragmented stack would incur unacceptable performance penalties for many workloads. Getting around that would require a ton of work, but maybe with hardware support, like automatically following the stack pointer to bring it into cache like a normal stack could get around that.
- duped 3y ago
  One problem is that when the stack is allocated it goes into the rest of the same virtual address space, and "growing" it might result in overlapping other address space allocated to the rest of the program. That's why there are guard pages around the stack (also, worth knowing that the stack doesn't grow up on some architectures, it can also grow down like on x86). That's why the post points out that growable stacks have to be moveable. The main solution is to just reserve a ton of virtual address space but avoid committing to it until the process actually writes to it, which is exactly what OS threads do. They reserve a large amount of virtual address space to start but it's more or less free until a thread actually uses it. However you may not see it released back to the OS until the process exits.
  - IshKebab 3y ago
    Yeah maybe an issue on 32-bit, but on 64-bit virtual address space is basically unlimited.
nick238 3y ago
Is the question "why async X" just short for "why cooperative multitasking in X language"? Coming from the Python world where I've written a ton of async code there (which I think is just a short way of saying cooperative concurrency?), it's generally easier to reason because you don't need to worry about getting preempted anywhere. If your application is mostly comprised of hurry-up-and-wait I/O code, single thread, single process can be great.
- harpiaharpyja 3y ago
  No, if you read the article the question is not "why you should use async in Rust" or anything like that. It's literally "why async Rust" as in "why async Rust is the thing that it is," why it is that way, how it came to be, etc. There's some unavoidable level of relationship, but that's not the focus of the article.
- zozbot234 3y ago
  Not quite, you can do cooperative multitasking using fibers ('stackful' coroutines).
codeflo 3y ago
> OS threads have a large pre-allocated stack, which increases per-thread memory overhead. It should be noted that AFAIK on all modern operating systems, only one page of the stack is actually allocated on thread creation, and the rest is merely reserved address space that will be allocated on demand. That doesn't make this point wrong: 4 KB is still a lot more than the couple of bytes a future might need. And setting up the page table is part of what makes spawning threads so expensive.
- vlovich123 3y ago
  That doesn’t really matter because in a long running process it’ll grow to use whatever your deepest call stack is. Matters less when you have few threads, matters a lot more when you have lots.
  - nyanpasu64 3y ago
    Can you have the runtime run a job every second, to trim thread stack depths and return memory to the kernel (whatever was meant by https://www.youtube.com/watch?v=kPR8h4-qZdk&t=1150s https://www.youtube.com/watch?v=kPR8h4-qZdk&t=1150s) while holding a FFI mutex (or a sharded one), and have every C function call first lock the mutex and expand the stack to the necessary depth?
    - scottlamb 3y ago
      Yes, you can use madvise(MADV_FREE) to return the RAM on Linux. I've seen a thread pool library that does this. I don't remember how this part works, but I think it's possible to maintain a high water mark in some inexpensive fashion to know when it's worth making this call.
      - vlovich123 3y ago
        Fucking with memory maps like that is super expensive and I wouldn’t advise it in any part that’s performance critical due to the cross CPU TLB flush that syscall will entail.
Dowwie 3y ago
Is anyone working on an async rust book? I'm aware of the draft that was last updated several years ago. I'm talking about a comprehensive, complete text on par with TRPL. The most difficult subject for rust programmers remains the least written about.
- steveklabnik 3y ago
  I am not aware of one, but I agree with you that it is sorely needed.
  - jph 3y ago
    Steve I'll donate $100 to whomever wants to lead it now.
  - echelon 3y ago
    Steve, could we sponsor you to write one or coordinate it? You've done so much great work on Rust education, documentation, community building, etc. I'd be happy to send money your way.
    - steveklabnik 3y ago
      I don't know if I have the energy in me. The Rust Project burned me out and continues to burn me out. Writing a book is a tremendous amount of work, and I am focused on my job over more broad open source contribution. That said, I appreciate the kind words. But I am struggling to have the energy to accept conference invitations and do smaller open source work, so I wouldn't commit to anything like that any time soon. (that said I literally arrived in Raleigh for All Things Open earlier today, where my talk will actually be about async/await, so... never say never.)
      - echelon 3y ago
        Totally understandable. You've worked like crazy for Rust. Everything you've done is greatly appreciated!
      - 3y ago
sujayakar 3y ago
as someone who's followed along (in large production systems) from `eventual` to a large combinator-based `futures`-0.1 system to async/await today, I really enjoyed this post and all its historical context. despite its complexity and slight ergonomic annoyances, async rust is a monumental achievement: safe userspace concurrency without heap allocation!
zeroCalories 3y ago
I've been feeling annoyed by async rust too, and have been brought back to Python for some of my side projects. That said, a lot of the cool features that I like from Rust feel tacked on in Python, so I've been looking at Ocaml. The ecosystem looks even worse than Rust, but it seems like it could be a solid replacement in terms of pure language features.
- mattnewport 3y ago
  Have you looked at F#? If you like the MLish aspects of Rust but want to not have to deal with some of the manual memory management complexity it's pretty nice. There's access to the whole .NET ecosystem too, as well as really good tooling and IDE support, though not all .NET libraries written in / for C# feel totally natural to consume from F#.
  - pjmlp 3y ago
    Some of them going forward aren't even possible to consume from F# unless there is a C# shim, e.g. anything that relies on code annoations for code generators, or post .NET 8, interceptors. OCaml is a much better option in the UNIX world.
lukebitts 3y ago
Great read! I always enjoy these dives into the reasoning behind the language
topspin 3y ago
The syntax isn't the problem. The syntax is fine: I've never had a serious problem with Rust's async syntax, and the language has been and continues to remove the papercuts that have emerged since async appeared. The thing about async Rust that absolutely destroys my efforts with it is the legion of problems involving dependencies and their async runtime peculiarities. As a result I've had to set Rust aside for most things where I'd otherwise love to employ it. I can't risk losing days to some tangle of an async runtime compatibility snafu. The one area that Rust has really astonished me recently is embedded Rust and MCUs: the ecosystem there is still young, but the results that can be produced with Rust in embedded development are really astonishing. Of course, there is typically no async runtime involved at that level, so problem solved...
- _ahxg 3y ago
  > I can't risk losing days to some tangle of an async runtime compatibility snafu. At this point in time, if you use Tokio, won't basically everything mainstream work? At least that has been my experience so far.
- pornel 3y ago
  The pain only exists if you're trying not to use Tokio that has 90% "market share" of Rust runtimes. I get that it is sometimes needed (if you use GTK or browser JS as your async runtime, or write your own executor). But for majority users there is only one runtime and there's no compatibility problem. Tokio won, and network effects killed everything else. Think of it like Golang's choice of runtimes (there isn't one).
  - p4ul 3y ago
    This is my impression, too! The door is open for other async runtimes, but for the majority of us, Tokio is the solution. And that's been just fine for me!
msvan 3y ago
I think the final remark about a hypothetical language, Rust-like but without all the low-level requirements, is important here. There is essentially no widely-adopted programming language out that feels like a modern ML with a good tooling situation. Until that happens, Rust will continue to awkwardly serve the audience of such a language while never truly being what they want it to be.
- evntdrvn 3y ago
  No love for F# ? The tooling is pretty dang good, all things considered.
  - pjmlp 3y ago
    It could be even better if despite being from Microsoft wasn't handled like a 3rd party guest language on .NET design decisions, and VS roadmap tooling.
    - evntdrvn 3y ago
      absolutely
- zozbot234 3y ago
  ReasonML is at least as widely adopted as any other niche language, and seems to fit the bill as a ML-like which plays nice with modern tooling.
- pohl 3y ago
  Why wouldn’t Swift fit that description?
  - tempodox 3y ago
    Like Rust, it's not an actual FP language. Borrowing some FP features is not sufficient.
    - 3y ago
egnehots 3y ago
As a Rust user, I have to say that it was an incredibly confusing feature. Not because of the technical decisions behind async/await, but because of the async ecosystem, especially the runtimes. Picture a Rust user when the feature came: - You can use it with future combinators! - No, actually, use it with a macro library that almost works but not really. - Here is Tokio (with a lot of moving parts). - Wait, async std is much simpler (again, almost seems to work but not really). - Wait, here is Smol, which is truly simpler (or at least smaller but not used). => You get "async fatigue." I can understand the position: let's not commit to anything before seeing what sticks on the wall. It might have worked for Serde (even that is debatable by some). But as a user, it's hard to follow, and you get the impression that this feature is not the stable foundation you can build on.
- littlestymaar 3y ago
  Honestly, it gets better once you learn to stop worrying and love tokio. I don't like the idea of having a single framework shaping up the entire Rust ecosystem, but at the same time once you just jump the tokio ship, everything just works out of the box and you don't have to worry. lots I wonder what are you referring to when talking about “lots of moving parts” when tokio has reached 1.0 three years ago and been pretty much stable since then.
  - dgacmu 3y ago
    Tokio is awesome and I end up using it on almost all of my server projects. But, particularly for library developers, one reason not to just jump on tokio and instead strive for (some, reasonable) compatibility across executors is the embedded world - async is an amazing match for programming tiny microprocessors because it provides an elegant syntactic sugar for all of those little interacting state machines, you would otherwise write, but you don't want a heavyweight thread-based executor to be mandatory. (The embassy embedded framework for rust is an example of this. It doesn't yet have as wide support as some non-async frameworks but for the things that it's compatible with, it's an absolute delight.)
    - 3y ago
demi56 3y ago
Honestly I see no reason why rust is so focused on Async if they’re C++ alternative (that is smaller runtime), having a unique keyword doesn’t seat right with me, I won’t be surprised if there will one day be a popular GC runtime
- steveklabnik 3y ago
  Just so you're aware, C++ has three keywords here: co_async, co_await, and co_return, so it feels a bit odd to complain about this aspect, specifically. Also, as I mentioned below, Rust fares even better than C++ on minimizing allocations here. C++ is used in high performing network services all the time, it shouldn't be a shock that Rust gets used for them as well.
  - demi56 3y ago
    Wow, so c++ is joining the gang too, looks like rust really is changing the programming ecosystem
    - steveklabnik 3y ago
      Rust was not the impetus for C++ getting this feature, just to be clear about it. Or at least, I never read that in any of the many papers over the years working through the design.
    - Buttons840 3y ago
      C# added async / await keywords first it seems, JavaScript a few years later. Rust is not the leader here.
  - kccqzy 3y ago
    C++ strongly depends on the actual implementation quality by leaving a lot of things to implementations. For example, the person leading the coroutine implementation in clang is very inconsistent on minimizing allocations than the GCC implementations. As an example they implemented a generator system using C++ coroutines that eventually compile down to a single instruction (https://godbolt.org/z/nsTjjGbn4 https://godbolt.org/z/nsTjjGbn4). As a counter argument though, clang's support for coroutines is still buggy and not ready for production use.
twic 3y ago
> I don’t know what to tell users who would rather just use threads and blocking IO. Certainly, I think there are a lot of systems for which that is a reasonable approach. And nothing in the Rust language prevents them from doing it. Their objection seems to be that the ecosystem on crates.io, especially for writing network services, is centered on using async/await. Ocassionally, I see a library which uses async/await in a “cargo cult” way, but mostly it seems safe to assume that the author of the library actually wants to perform non-blocking IO and get the performance benefits of user-space concurrency. This seems like a glib dismissal of a real problem. If you want to do threads-and-blocking programming in the large in Rust, you basically can't, because async has become a very strong default in the ecosystem. And i don't think anyone could look at the ecosystem and honestly form the opinion that it's that way because every library developer has made the informed decision that their use-case requires async, when use cases which require async are so incredibly rare - it is absolutely a social default. > None of us can control what everyone else decides to work on, and the fact of the matter is just that most people who release networking-related libraries on crates.io want to use async Rust, whether for business reasons or just out of interest. I’d like it to be easier to use those libraries in a non-async context (e.g. by bringing a pollster-like API into the standard library), but it’s hard to know what to say to people who’s gripe is that the people putting code online for free don’t have exactly the same use case as them. The criticism is not about the library authors, but about the stewards of the language and community who laid the path for them to follow.
- galangalalgol 3y ago
  Ok, I feel a bit dense here, but I have an application that has threads and blocking. Thread pools for processing bulk data, and tokio for serving up the results. Am I missing something?
- bryanlarsen 3y ago
  I believe your criticism is answered by one of the proposals in the post: to pull the 'pollster' crate or something similar into the standard library. It's not quite "colorless", but it's practical. Pulling pollster into the standard library is a very minor change technically, but is much larger socially. The blessing of pollster will make blocking a first class citizen again.
jrpelkonen 3y ago
This is an excellent post about the evolution and the milestones that lead to the current async Rust approach. I thought the technical arguments were clearly articulated, dispassionately responding to the recent critical viewpoints regarding async Rust. I hope that people, especially the ones that have voiced these criticisms, take the time to read and digest this article. It may not change their mind, but perhaps it will help them understand the current situation better. Unfortunately, there are already some comments that prove my optimism misguided elsewhere in this thread.
- soerxpso 3y ago
  Although the articles arguments show that the decisions made were understandable and perhaps the "right decisions" for the language, there's a common theme that backwards compatibility was a motivator. I'm not saying they should have broken backwards compatibility, but "This is actually the best way to do it," and, "This is the best way that we could do it given our tough situation," are two very different points. I don't really care about the latter when I'm evaluating whether I actually like the way it's done.
  - withoutboats3 3y ago
    The only thing constrained by backward compatibility was using Pin instead of Move. Async/await itself is the natural progression for Rust for the reasons I outlined at length regarding how Rust represents zero-cost coroutines.
Buttons840 3y ago
I used Python's Twisted framework back in 2008 or so. This is before any language had async / await keywords (of course, the concept was there, just not the syntax level support). Twisted built async on top of Python's generator functions, as far as I understand. I see now that the Rust community talks about supporting generators, and I've wondered if Rust did things backwards. If Rust had received generators first, could async have been built on top of generators without async specific special syntax?
- pornel 3y ago
  Rust had generators first and used them to experiment with async. They're still there, stuck in nightly/unstable. https://doc.rust-lang.org/beta/unstable-book/language-features/generators.html https://doc.rust-lang.org/beta/unstable-book/language-featur... I think that's because there was a huge demand for async specifically, and Rust could ship higher level async without solving all the design details of less desired generators first.
  - Buttons840 3y ago
    I didn't know that, thanks for telling me.
troppl 3y ago
I have a question that maybe someone here knows. It has been noted multiple times in the comments here that at this point, one should just use tokio as the async runtime. (From my limited experience, I agree) The async Rust book [0] says this about runtimes: > Importantly, executors, tasks, reactors, combinators, and low-level I/O futures and traits are not yet provided in the standard library. In the meantime, community-provided async ecosystems fill in these gaps. Notably, it says "not yet". My question is if someone knows if there are actual plans to incorporate any (existing) async runtime, and if so, whether there is a timeline? Also, is tokio in the talks to be the runtime, or is this still open? [0]: https://rust-lang.github.io/async-book/08_ecosystem/00_chapter.html https://rust-lang.github.io/async-book/08_ecosystem/00_chapt...
- afdbcreid 3y ago
  There are no plans to incorporate a big async runtime like tokio. This does not matches Rust's vision of small, well-focused std. There are possible plans to implement a minimal runtime that only allows you to execute futures. No I/O or such. Mainly for tests or examples. No timeline though, at least as far as I'm aware. There are also plans to standardize the async traits, e.g. spawn a new task, AsyncRead and AsyncWrite etc.. I don't think there is a timeline, but they wait at least for async fn in traits.
  - scottlamb 3y ago
    > I don't think there is a timeline, but they wait at least for async fn in traits. That was just merged! [1] It should be stable in 1.75 on December 28th. [1] https://github.com/rust-lang/rust/pull/115822 https://github.com/rust-lang/rust/pull/115822
    - afdbcreid 3y ago
      I know it was merged, but currently there is only support for statically-dispatched async fn in traits. I believe we also need dynamic dispatch to compete with the current style of `poll_X()` functions in the I/O traits.
randyrand 3y ago
another async approach that i don’t see mentioned often is what apple chose with Grand Central Dispatch i.e pre allocated worker threads and work queues perhaps all we needed was some nice semantics around that
- yencabulator 3y ago
  See glommio and tokio-uring. Unfortunately not stable, not polished at all, and receiving a fraction of the attention the main Tokio API is getting. The biggest pain coming up is how io_uring changes the shape of the read/write APIs everywhere :-/ https://www.datadoghq.com/blog/engineering/introducing-glommio/ https://www.datadoghq.com/blog/engineering/introducing-glomm... https://itnext.io/modern-storage-is-plenty-fast-it-is-the-apis-that-are-bad-6a68319fbc1a https://itnext.io/modern-storage-is-plenty-fast-it-is-the-ap... https://news.ycombinator.com/item?id=25220892 https://news.ycombinator.com/item?id=25220892 https://www.reddit.com/r/rust/comments/k16j6x/modern_storage_is_plenty_fast_it_is_the_apis_that/ https://www.reddit.com/r/rust/comments/k16j6x/modern_storage... https://www.reddit.com/r/programming/comments/k0yyk7/modern_storage_is_plenty_fast_it_is_the_apis_that/ https://www.reddit.com/r/programming/comments/k0yyk7/modern_... https://www.youtube.com/watch?v=PbgTyCSDPrs https://www.youtube.com/watch?v=PbgTyCSDPrs https://github.com/tokio-rs/tokio-uring/ https://github.com/tokio-rs/tokio-uring/
bryanlarsen 3y ago
> I’d like it to be easier to use those libraries in a non-async context (e.g. by bringing a pollster-like API into the standard library), I think this would simultaneously solve two of the major gripes expressed here and elsewhere. 1) An easy answer for those who wish to avoid being sucked into async just because there's a useful crate that has async. 2) Blessing an executor other than tokio avoids tokio lock in.

212 points by withoutboats3 3y ago

A lot of discussion about async Rust assumes that the reason one would want to use async/Futures is for performance and scalability reasons. Personally, though, I would strongly prefer to use async rather than explicit threading even for cases where performance wasn’t the highest priority. The conceptual model is just better. Futures allow you to cleanly express composition of sub-tasks in a way that explicit threading doesn’t: https://monkey.org/~marius/futures-arent-ersatz-threads.html https://monkey.org/~marius/futures-arent-ersatz-threads.html

riku_iki 3y ago
Futures as a concept are orthogonal to async, they can totally work in explicit threading model.
- _ahxg 3y ago
  Do you know an example of such case in the Rust ecosystem?
  - riku_iki 3y ago
    No, I don't know, I am talking about general concept. In Java, Future.get() blocks current thread, and it is trivially integrated into explicit threading programming. In Rust, Future.poll() is not blocking, and one would need to rely on some async framework, or build own event loop which can potentially block thread.
  - tux3 3y ago
    It's worth noting that a thread's JoinHandle provides a similar interface. You can spawn your tasks, store the JoinHandle "futures", and wait for completion whenever you need the result. A difference being that Futures do nothing until polled, while threads start on their own, but that's arguably a helpful simplification for this purpose.
    - convolvatron 3y ago
      this I really found disconcerting - the model I often want is I want to start some work, and then join at some later point - or even chain directly into the next task. but instead I start a future, and then to run it at all I need to wait for the result. I understand the there are tools to effect this, but it really leaves you wondering - what did I just do? start an async task and then .. block on it in order to get it to execute?
      - laurencerowe 3y ago
        I'm not an expert, but my understanding is that Rust makes a distinction between Futures and Tasks, the top level Futures that are run by the executor. In JavaScript terms Futures are more like sugar around callbacks, they don't do anything until you call/poll them. Tasks are independent entities like Promises which are being run by the executor, though they may currently be blocked on other tasks. > the model I often want is I want to start some work, and then join at some later point - or even chain directly into the next task. Rust wants you to do this the other way around. First chain together your futures so that when you start the top level one as a task there is a single state machine for it to run.
- jupp0r 3y ago
  They can be used that way, but you end up with exactly the same problems that async programming aims to avoid (performance, deadlocks, your business logic being cluttered with low level implementation details).
  - riku_iki 3y ago
    > that async programming aims to avoid (performance, deadlocks, your business logic being cluttered with low level implementation details). I disagree with you, my code looks safe and simple with explicit blocking threading, and at the same time is much simpler to reason about what is going on and tune in contrast to async frameworks which hide most of the details under the hood. You can argue about performance, that async/epoll/etc allows to avoid spawning thousands of threads and remove some overhead, but there is no much benchmarks in internet (per my research) which would say that this performance overhead is large.
    - jupp0r 3y ago
      If you are using explicit blocking, share data between threads and have not run into deadlocks then your application is trivial (which is great if it solves your problem).
      - riku_iki 3y ago
        Could you explain how sharing data between threads is different in async programming and blocking programming?
        jupp0r 3y ago
        You can minimize sharing data between threads because it's easier to have data affinity with threads (ie only thread A will read or write to a piece of data). You can still access that data from multiple modules because the whole thread is never blocked waiting for IO (because of async). An extreme example is nodejs, where you only have one thread, can concurrently do thousands of things and never have to coordinate (ie via mutexes) data access.
ori_b 3y ago
Futures are nicely equivalent to one shot channels with threads.
- littlestymaar 3y ago
  With a nice syntax sugar on top, but yeah pretty much.
- pornel 3y ago
  Not quite: Rust futures also have immediate cancellation and easy timeouts that can be imposed externally on any future. In threads that perform blocking I/O you don't get that, and need to support timeouts and cancellation explicitly in every blocking call.

Animats 3y ago

He gives the business case: It was clear, though usually left unsaid, that what Rust needed to succeed was industry adoption, so that it could continue to receive support once Mozilla stopped being willing to fund an experimental new language. And it was clear that the most likely path to short-term industry adoption was in network services, especially those with a performance profile that compelled them at the time to be written in C/C++ ... The other advantage of network services was that this wing of the software industry has the flexibility and appetite to rapidly adopt a new technology like Rust. The other domains were - and are! - viable long term opportunities for Rust, but they were seen as not as quick to adopt new technology (embedded), depended on a new platform that had not yet seen widespread adoption itself (WebAssembly), or were not a particularly lucrative industrial application that could lead to funding for the language (CLIs). I drove at async/await with the diligent fervor of the assumption that Rust’s survival depended on this feature. ... Many of the most prominent sponsors of the Rust Foundation, especially those who pay developers, depend on async/await to write high performance network services in Rust as one of their primary use cases that justify their funding. Using async/await for embedded systems or kernel programming is also a growing area of interest with a bright future. Async/await has been so successful that the most common complaint about it is that the ecosystem is too centered on it, rather than “normal” Rust. I don’t know what to tell users who would rather just use threads and blocking IO. Certainly, I think there are a lot of systems for which that is a reasonable approach. And nothing in the Rust language prevents them from doing it. Their objection seems to be that the ecosystem on crates.io, especially for writing network services, is centered on using async/await. ... None of us can control what everyone else decides to work on, and the fact of the matter is just that most people who release networking-related libraries on crates.io want to use async Rust, whether for business reasons or just out of interest. I’d like it to be easier to use those libraries in a non-async context (e.g. by bringing a pollster-like API into the standard library), but it’s hard to know what to say to people who’s gripe is that the people putting code online for free don’t have exactly the same use case as them. Well, that says it. Rust has pivoted to web stuff. For which Go is probably better suited. Good-bye, Rust as a systems language, or for game development. Younger programmers will probably still be seeing C/C++ buffer overflows in 2050. The technical problem is that pure async, like JavaScript, is fine, and pure threading, like classic Rust, is fine. But the combination is awful.

IshKebab 3y ago

> Context-switching between the kernel and userspace is expensive in terms of CPU cycles. > OS threads have a large pre-allocated stack, which increases per-thread memory overhead. It feels like it would be totally possibly to improve OSes to remove this limitation. Is anyone actually working on that? For example I don't see why you couldn't have growable stacks for threads. Or have first class hardware support for context switching. (Yes that would take a long time to arrive.)

zeroCalories 3y ago
> It feels like it would be totally possibly to improve OSes to remove this limitation. Is anyone actually working on that? This is just my guess, but I assume they don't use growable stacks for the same reason Rust doesn't. C doesn't have a garbage collector, and making a fragmented stack would incur unacceptable performance penalties for many workloads. Getting around that would require a ton of work, but maybe with hardware support, like automatically following the stack pointer to bring it into cache like a normal stack could get around that.
duped 3y ago
One problem is that when the stack is allocated it goes into the rest of the same virtual address space, and "growing" it might result in overlapping other address space allocated to the rest of the program. That's why there are guard pages around the stack (also, worth knowing that the stack doesn't grow up on some architectures, it can also grow down like on x86). That's why the post points out that growable stacks have to be moveable. The main solution is to just reserve a ton of virtual address space but avoid committing to it until the process actually writes to it, which is exactly what OS threads do. They reserve a large amount of virtual address space to start but it's more or less free until a thread actually uses it. However you may not see it released back to the OS until the process exits.
- IshKebab 3y ago
  Yeah maybe an issue on 32-bit, but on 64-bit virtual address space is basically unlimited.

nick238 3y ago

Is the question "why async X" just short for "why cooperative multitasking in X language"? Coming from the Python world where I've written a ton of async code there (which I think is just a short way of saying cooperative concurrency?), it's generally easier to reason because you don't need to worry about getting preempted anywhere. If your application is mostly comprised of hurry-up-and-wait I/O code, single thread, single process can be great.

harpiaharpyja 3y ago
No, if you read the article the question is not "why you should use async in Rust" or anything like that. It's literally "why async Rust" as in "why async Rust is the thing that it is," why it is that way, how it came to be, etc. There's some unavoidable level of relationship, but that's not the focus of the article.
zozbot234 3y ago
Not quite, you can do cooperative multitasking using fibers ('stackful' coroutines).

codeflo 3y ago

> OS threads have a large pre-allocated stack, which increases per-thread memory overhead. It should be noted that AFAIK on all modern operating systems, only one page of the stack is actually allocated on thread creation, and the rest is merely reserved address space that will be allocated on demand. That doesn't make this point wrong: 4 KB is still a lot more than the couple of bytes a future might need. And setting up the page table is part of what makes spawning threads so expensive.

vlovich123 3y ago
That doesn’t really matter because in a long running process it’ll grow to use whatever your deepest call stack is. Matters less when you have few threads, matters a lot more when you have lots.
- nyanpasu64 3y ago
  Can you have the runtime run a job every second, to trim thread stack depths and return memory to the kernel (whatever was meant by https://www.youtube.com/watch?v=kPR8h4-qZdk&t=1150s https://www.youtube.com/watch?v=kPR8h4-qZdk&t=1150s) while holding a FFI mutex (or a sharded one), and have every C function call first lock the mutex and expand the stack to the necessary depth?
  - scottlamb 3y ago
    Yes, you can use madvise(MADV_FREE) to return the RAM on Linux. I've seen a thread pool library that does this. I don't remember how this part works, but I think it's possible to maintain a high water mark in some inexpensive fashion to know when it's worth making this call.
    - vlovich123 3y ago
      Fucking with memory maps like that is super expensive and I wouldn’t advise it in any part that’s performance critical due to the cross CPU TLB flush that syscall will entail.

Dowwie 3y ago

Is anyone working on an async rust book? I'm aware of the draft that was last updated several years ago. I'm talking about a comprehensive, complete text on par with TRPL. The most difficult subject for rust programmers remains the least written about.

steveklabnik 3y ago
I am not aware of one, but I agree with you that it is sorely needed.
- jph 3y ago
  Steve I'll donate $100 to whomever wants to lead it now.
- echelon 3y ago
  Steve, could we sponsor you to write one or coordinate it? You've done so much great work on Rust education, documentation, community building, etc. I'd be happy to send money your way.
  - steveklabnik 3y ago
    I don't know if I have the energy in me. The Rust Project burned me out and continues to burn me out. Writing a book is a tremendous amount of work, and I am focused on my job over more broad open source contribution. That said, I appreciate the kind words. But I am struggling to have the energy to accept conference invitations and do smaller open source work, so I wouldn't commit to anything like that any time soon. (that said I literally arrived in Raleigh for All Things Open earlier today, where my talk will actually be about async/await, so... never say never.)
    - echelon 3y ago
      Totally understandable. You've worked like crazy for Rust. Everything you've done is greatly appreciated!
    - 3y ago

sujayakar 3y ago

as someone who's followed along (in large production systems) from `eventual` to a large combinator-based `futures`-0.1 system to async/await today, I really enjoyed this post and all its historical context. despite its complexity and slight ergonomic annoyances, async rust is a monumental achievement: safe userspace concurrency without heap allocation!

zeroCalories 3y ago

I've been feeling annoyed by async rust too, and have been brought back to Python for some of my side projects. That said, a lot of the cool features that I like from Rust feel tacked on in Python, so I've been looking at Ocaml. The ecosystem looks even worse than Rust, but it seems like it could be a solid replacement in terms of pure language features.

mattnewport 3y ago
Have you looked at F#? If you like the MLish aspects of Rust but want to not have to deal with some of the manual memory management complexity it's pretty nice. There's access to the whole .NET ecosystem too, as well as really good tooling and IDE support, though not all .NET libraries written in / for C# feel totally natural to consume from F#.
- pjmlp 3y ago
  Some of them going forward aren't even possible to consume from F# unless there is a C# shim, e.g. anything that relies on code annoations for code generators, or post .NET 8, interceptors. OCaml is a much better option in the UNIX world.

lukebitts 3y ago

Great read! I always enjoy these dives into the reasoning behind the language

topspin 3y ago

The syntax isn't the problem. The syntax is fine: I've never had a serious problem with Rust's async syntax, and the language has been and continues to remove the papercuts that have emerged since async appeared. The thing about async Rust that absolutely destroys my efforts with it is the legion of problems involving dependencies and their async runtime peculiarities. As a result I've had to set Rust aside for most things where I'd otherwise love to employ it. I can't risk losing days to some tangle of an async runtime compatibility snafu. The one area that Rust has really astonished me recently is embedded Rust and MCUs: the ecosystem there is still young, but the results that can be produced with Rust in embedded development are really astonishing. Of course, there is typically no async runtime involved at that level, so problem solved...

_ahxg 3y ago
> I can't risk losing days to some tangle of an async runtime compatibility snafu. At this point in time, if you use Tokio, won't basically everything mainstream work? At least that has been my experience so far.
pornel 3y ago
The pain only exists if you're trying not to use Tokio that has 90% "market share" of Rust runtimes. I get that it is sometimes needed (if you use GTK or browser JS as your async runtime, or write your own executor). But for majority users there is only one runtime and there's no compatibility problem. Tokio won, and network effects killed everything else. Think of it like Golang's choice of runtimes (there isn't one).
- p4ul 3y ago
  This is my impression, too! The door is open for other async runtimes, but for the majority of us, Tokio is the solution. And that's been just fine for me!

msvan 3y ago

I think the final remark about a hypothetical language, Rust-like but without all the low-level requirements, is important here. There is essentially no widely-adopted programming language out that feels like a modern ML with a good tooling situation. Until that happens, Rust will continue to awkwardly serve the audience of such a language while never truly being what they want it to be.

evntdrvn 3y ago
No love for F# ? The tooling is pretty dang good, all things considered.
- pjmlp 3y ago
  It could be even better if despite being from Microsoft wasn't handled like a 3rd party guest language on .NET design decisions, and VS roadmap tooling.
  - evntdrvn 3y ago
    absolutely
zozbot234 3y ago
ReasonML is at least as widely adopted as any other niche language, and seems to fit the bill as a ML-like which plays nice with modern tooling.
pohl 3y ago
Why wouldn’t Swift fit that description?
- tempodox 3y ago
  Like Rust, it's not an actual FP language. Borrowing some FP features is not sufficient.
  - 3y ago

egnehots 3y ago

As a Rust user, I have to say that it was an incredibly confusing feature. Not because of the technical decisions behind async/await, but because of the async ecosystem, especially the runtimes. Picture a Rust user when the feature came: - You can use it with future combinators! - No, actually, use it with a macro library that almost works but not really. - Here is Tokio (with a lot of moving parts). - Wait, async std is much simpler (again, almost seems to work but not really). - Wait, here is Smol, which is truly simpler (or at least smaller but not used). => You get "async fatigue." I can understand the position: let's not commit to anything before seeing what sticks on the wall. It might have worked for Serde (even that is debatable by some). But as a user, it's hard to follow, and you get the impression that this feature is not the stable foundation you can build on.

littlestymaar 3y ago
Honestly, it gets better once you learn to stop worrying and love tokio. I don't like the idea of having a single framework shaping up the entire Rust ecosystem, but at the same time once you just jump the tokio ship, everything just works out of the box and you don't have to worry. lots I wonder what are you referring to when talking about “lots of moving parts” when tokio has reached 1.0 three years ago and been pretty much stable since then.
- dgacmu 3y ago
  Tokio is awesome and I end up using it on almost all of my server projects. But, particularly for library developers, one reason not to just jump on tokio and instead strive for (some, reasonable) compatibility across executors is the embedded world - async is an amazing match for programming tiny microprocessors because it provides an elegant syntactic sugar for all of those little interacting state machines, you would otherwise write, but you don't want a heavyweight thread-based executor to be mandatory. (The embassy embedded framework for rust is an example of this. It doesn't yet have as wide support as some non-async frameworks but for the things that it's compatible with, it's an absolute delight.)
  - 3y ago

demi56 3y ago

Honestly I see no reason why rust is so focused on Async if they’re C++ alternative (that is smaller runtime), having a unique keyword doesn’t seat right with me, I won’t be surprised if there will one day be a popular GC runtime

steveklabnik 3y ago
Just so you're aware, C++ has three keywords here: co_async, co_await, and co_return, so it feels a bit odd to complain about this aspect, specifically. Also, as I mentioned below, Rust fares even better than C++ on minimizing allocations here. C++ is used in high performing network services all the time, it shouldn't be a shock that Rust gets used for them as well.
- demi56 3y ago
  Wow, so c++ is joining the gang too, looks like rust really is changing the programming ecosystem
  - steveklabnik 3y ago
    Rust was not the impetus for C++ getting this feature, just to be clear about it. Or at least, I never read that in any of the many papers over the years working through the design.
  - Buttons840 3y ago
    C# added async / await keywords first it seems, JavaScript a few years later. Rust is not the leader here.
- kccqzy 3y ago
  C++ strongly depends on the actual implementation quality by leaving a lot of things to implementations. For example, the person leading the coroutine implementation in clang is very inconsistent on minimizing allocations than the GCC implementations. As an example they implemented a generator system using C++ coroutines that eventually compile down to a single instruction (https://godbolt.org/z/nsTjjGbn4 https://godbolt.org/z/nsTjjGbn4). As a counter argument though, clang's support for coroutines is still buggy and not ready for production use.

twic 3y ago

> I don’t know what to tell users who would rather just use threads and blocking IO. Certainly, I think there are a lot of systems for which that is a reasonable approach. And nothing in the Rust language prevents them from doing it. Their objection seems to be that the ecosystem on crates.io, especially for writing network services, is centered on using async/await. Ocassionally, I see a library which uses async/await in a “cargo cult” way, but mostly it seems safe to assume that the author of the library actually wants to perform non-blocking IO and get the performance benefits of user-space concurrency. This seems like a glib dismissal of a real problem. If you want to do threads-and-blocking programming in the large in Rust, you basically can't, because async has become a very strong default in the ecosystem. And i don't think anyone could look at the ecosystem and honestly form the opinion that it's that way because every library developer has made the informed decision that their use-case requires async, when use cases which require async are so incredibly rare - it is absolutely a social default. > None of us can control what everyone else decides to work on, and the fact of the matter is just that most people who release networking-related libraries on crates.io want to use async Rust, whether for business reasons or just out of interest. I’d like it to be easier to use those libraries in a non-async context (e.g. by bringing a pollster-like API into the standard library), but it’s hard to know what to say to people who’s gripe is that the people putting code online for free don’t have exactly the same use case as them. The criticism is not about the library authors, but about the stewards of the language and community who laid the path for them to follow.

galangalalgol 3y ago
Ok, I feel a bit dense here, but I have an application that has threads and blocking. Thread pools for processing bulk data, and tokio for serving up the results. Am I missing something?
bryanlarsen 3y ago
I believe your criticism is answered by one of the proposals in the post: to pull the 'pollster' crate or something similar into the standard library. It's not quite "colorless", but it's practical. Pulling pollster into the standard library is a very minor change technically, but is much larger socially. The blessing of pollster will make blocking a first class citizen again.

jrpelkonen 3y ago

This is an excellent post about the evolution and the milestones that lead to the current async Rust approach. I thought the technical arguments were clearly articulated, dispassionately responding to the recent critical viewpoints regarding async Rust. I hope that people, especially the ones that have voiced these criticisms, take the time to read and digest this article. It may not change their mind, but perhaps it will help them understand the current situation better. Unfortunately, there are already some comments that prove my optimism misguided elsewhere in this thread.

soerxpso 3y ago
Although the articles arguments show that the decisions made were understandable and perhaps the "right decisions" for the language, there's a common theme that backwards compatibility was a motivator. I'm not saying they should have broken backwards compatibility, but "This is actually the best way to do it," and, "This is the best way that we could do it given our tough situation," are two very different points. I don't really care about the latter when I'm evaluating whether I actually like the way it's done.
- withoutboats3 3y ago
  The only thing constrained by backward compatibility was using Pin instead of Move. Async/await itself is the natural progression for Rust for the reasons I outlined at length regarding how Rust represents zero-cost coroutines.

Buttons840 3y ago

I used Python's Twisted framework back in 2008 or so. This is before any language had async / await keywords (of course, the concept was there, just not the syntax level support). Twisted built async on top of Python's generator functions, as far as I understand. I see now that the Rust community talks about supporting generators, and I've wondered if Rust did things backwards. If Rust had received generators first, could async have been built on top of generators without async specific special syntax?

pornel 3y ago
Rust had generators first and used them to experiment with async. They're still there, stuck in nightly/unstable. https://doc.rust-lang.org/beta/unstable-book/language-features/generators.html https://doc.rust-lang.org/beta/unstable-book/language-featur... I think that's because there was a huge demand for async specifically, and Rust could ship higher level async without solving all the design details of less desired generators first.
- Buttons840 3y ago
  I didn't know that, thanks for telling me.

troppl 3y ago

I have a question that maybe someone here knows. It has been noted multiple times in the comments here that at this point, one should just use tokio as the async runtime. (From my limited experience, I agree) The async Rust book [0] says this about runtimes: > Importantly, executors, tasks, reactors, combinators, and low-level I/O futures and traits are not yet provided in the standard library. In the meantime, community-provided async ecosystems fill in these gaps. Notably, it says "not yet". My question is if someone knows if there are actual plans to incorporate any (existing) async runtime, and if so, whether there is a timeline? Also, is tokio in the talks to be the runtime, or is this still open? [0]: https://rust-lang.github.io/async-book/08_ecosystem/00_chapter.html https://rust-lang.github.io/async-book/08_ecosystem/00_chapt...

afdbcreid 3y ago
There are no plans to incorporate a big async runtime like tokio. This does not matches Rust's vision of small, well-focused std. There are possible plans to implement a minimal runtime that only allows you to execute futures. No I/O or such. Mainly for tests or examples. No timeline though, at least as far as I'm aware. There are also plans to standardize the async traits, e.g. spawn a new task, AsyncRead and AsyncWrite etc.. I don't think there is a timeline, but they wait at least for async fn in traits.
- scottlamb 3y ago
  > I don't think there is a timeline, but they wait at least for async fn in traits. That was just merged! [1] It should be stable in 1.75 on December 28th. [1] https://github.com/rust-lang/rust/pull/115822 https://github.com/rust-lang/rust/pull/115822
  - afdbcreid 3y ago
    I know it was merged, but currently there is only support for statically-dispatched async fn in traits. I believe we also need dynamic dispatch to compete with the current style of `poll_X()` functions in the I/O traits.

randyrand 3y ago

another async approach that i don’t see mentioned often is what apple chose with Grand Central Dispatch i.e pre allocated worker threads and work queues perhaps all we needed was some nice semantics around that

yencabulator 3y ago
See glommio and tokio-uring. Unfortunately not stable, not polished at all, and receiving a fraction of the attention the main Tokio API is getting. The biggest pain coming up is how io_uring changes the shape of the read/write APIs everywhere :-/ https://www.datadoghq.com/blog/engineering/introducing-glommio/ https://www.datadoghq.com/blog/engineering/introducing-glomm... https://itnext.io/modern-storage-is-plenty-fast-it-is-the-apis-that-are-bad-6a68319fbc1a https://itnext.io/modern-storage-is-plenty-fast-it-is-the-ap... https://news.ycombinator.com/item?id=25220892 https://news.ycombinator.com/item?id=25220892 https://www.reddit.com/r/rust/comments/k16j6x/modern_storage_is_plenty_fast_it_is_the_apis_that/ https://www.reddit.com/r/rust/comments/k16j6x/modern_storage... https://www.reddit.com/r/programming/comments/k0yyk7/modern_storage_is_plenty_fast_it_is_the_apis_that/ https://www.reddit.com/r/programming/comments/k0yyk7/modern_... https://www.youtube.com/watch?v=PbgTyCSDPrs https://www.youtube.com/watch?v=PbgTyCSDPrs https://github.com/tokio-rs/tokio-uring/ https://github.com/tokio-rs/tokio-uring/

bryanlarsen 3y ago

> I’d like it to be easier to use those libraries in a non-async context (e.g. by bringing a pollster-like API into the standard library), I think this would simultaneously solve two of the major gripes expressed here and elsewhere. 1) An easy answer for those who wish to avoid being sucked into async just because there's a useful crate that has async. 2) Blessing an executor other than tokio avoids tokio lock in.