6 ms·

Async Rust in Three Parts

162 points by oconnor663 2y ago

Animats 2y ago
It's amusing that the Rust Playground lets you run a hundred threads. That's generous of them. There's a ceiling below 1000, though. The author points out, "On my Linux laptop I can spawn almost 19k threads before I hit this crash, but the Playground has tighter resource limits." Large servers can go higher than that. The thread-based I/O example with the compute bound poll loop is kind of strange. "Join" isn't really that useful when you have unrelated threads running finite tasks. Usually, you let the thread do its thing,finish, put results on a queue, and let the thread exit. Then it doesn't matter who finishes first. Rust join is actually optional. You don't have to join to close out a thread and reap the thread resources. It's not like zombies in Unix/Linux, where some resources are tied up until the parent process calls wait(). Loops where you join all the threads that are supposedly finished are usually troublesome. If somebody gets stuck, the joiner stalls. Clean exits from programs with lots of channels are troublesome. Channels with multiple senders don't close until all senders exit, which can be hard to arrange when something detects an error. In Rust, the main thread is special. (I consider this unfortunate, but web people like it, because inside browsers, the main thread is very special.) If the main thread exits, all the other threads are silently killed.
- diath 2y ago
  > Loops where you join all the threads that are supposedly finished are usually troublesome. If somebody gets stuck, the joiner stalls. That makes sense if the main thread is actually doing any useful work, but when its only job is to spawn threads and wait for them to finish before exiting, then it's a pretty common idiom.
- willglynn 2y ago
  > In Rust, the main thread is special. (I consider this unfortunate, but web people like it, because inside browsers, the main thread is very special.) If the main thread exits, all the other threads are silently killed. Rust inherits this from `pthread_detach()`: The detached attribute merely determines the behavior of the system when the thread terminates; it does not prevent the thread from being terminated if the process terminates using exit(3) (or equivalently, if the main thread returns).
  - wahern 2y ago
    The main thread is special because that's how the runtime works on Unix. In particular, when "main" exits, the process exits. This is required by the C standard. It's also fundamentally built into how Unix processes work, as certain global variables, like argv and environ strings, typically are stored on the main thread's stack, so if the main thread is destroyed those references become invalid. In principle Rust could have defined its environment to not make the main thread special, but then it would need some additional runtime magic on Unix systems, including having the main thread poll for all other threads to exit, which in turn would require it to add a layer of indirection to the system's threading runtime (e.g. wrapping pthreads) to be able to track all threads.
    - kelnos 2y ago
      > In principle Rust could have defined its environment to not make the main thread special... Not to mention they'd have to be very careful with what they do on the main thread after they start up the application's first thread (e.g. allocating memory via malloc() is out), since there are quite a few things that are not safe to do (like fork() that's not immediately followed by exec()) in a multi-threaded program. So even a "single-threaded" Rust program would become multi-threaded, and assume all those problems.
- oconnor663 2y ago
  > Rust join is actually optional. I was recently surprised to learn that returning from main() with background threads still running is more or less UB in C++, because those threads can race against static destructors: https://www.reddit.com/r/cpp/comments/1fu0y6n/when_a_background_thread_races_against_the https://www.reddit.com/r/cpp/comments/1fu0y6n/when_a_backgro.... C doesn't have this issue, though, as far as I know?
  - dwattttt 2y ago
    atexit enters the chat
  - loeg 2y ago
    Common C implementations (clang, gcc) have static destructors as an extension, though C codebases probably use this a lot less than C++ ones with static objects and destructor methods.
- shepmaster 2y ago
  > the Rust Playground lets you run a hundred threads It's more that we don't do anything to prevent it, other than coarse process-wide memory / CPU time limits. IIRC, Rust-spawned threads on Linux use 2MiB of stack space by default, so that seems like a likely cap. Note that the playground is only 2 cores and you are sharing with everyone else, so you aren't likely to really benefit.
  - littlestymaar 2y ago
    > Note that the playground is only 2 cores and you are sharing with everyone else This is amazing, I use it all the time with no performance issues so I expected it to be much beefier to support many simultaneous users. How many users does it serve? (Monthly or daily user and/or compilation job sent). And what tricks are used to keep it working? (I suspect it can re-use already compiled binaries of all supported dependencies and only need to compile the user's code and link it, but is there other clever strategies?)
    - shepmaster 2y ago
      > How many users does it serve? I don't really track users, but over the last 24 hours, there were 47.8k meaningful [1] requests taking a total of 28.2 hours. That ~0.5 requests per second number has been relatively consistent. > re-use already compiled binaries of all supported dependencies and only need to compile the user's code and link it, but is there other clever strategies? Yes, we pre-compile all the available dependencies [2] and that's about it. > I use it all the time with no performance issues That's good to hear! There's been a long-running bug where the playground binary loses track of the child Docker container (maybe?) and then the machine runs out of memory and the OOM killer often does more harm than good [3]. While trying to pin that down, I've recently caused the entire process to get into what appears to be a complete deadlock where no requests can be serviced at all. This tends to happen while I'm asleep so either I have no chance to debug it before it is auto-killed or the playground is unresponsive for 8+ hours. [1]: compiling / executing code, running clippy/miri/rustfmt, expanding macros [2]: https://github.com/rust-lang/rust-playground/blob/c4d00b90aab3742c841011b29fba038f3cd99690/compiler/base/Dockerfile#L105-L107 https://github.com/rust-lang/rust-playground/blob/c4d00b90aa... [3]: somehow it does something that ends up killing the network stack and then the machine is basically dead in the water. Very similar to what is reported in https://serverfault.com/q/1125634/119136 https://serverfault.com/q/1125634/119136
- dwattttt 2y ago
  > Loops where you join all the threads that are supposedly finished are usually troublesome. If somebody gets stuck, the joiner stalls. Clean exits from programs with lots of channels are troublesome. Channels with multiple senders don't close until all senders exit, which can be hard to arrange when something detects an error. I wish join-with-timeout was a more common/supported operation.
alilleybrinker 2y ago
In part two, the author explains trait objects in a way that is, I think, a little misleading. They're right that trait objects are dynamically sized types, which means they can't be passed by value to functions, but wrong that they need to be boxed; they can instead be put behind a reference. Both of the following are valid types. type DynFutureBox = Pin<Box<dyn Future<Output = ()>>>; type DynFutureRef<'f> = Pin<&'f dyn Future<Output = ()>>; You can see this in the Rust Playground here: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=6c61667f75dfc2435e6a342f897d1c30 https://play.rust-lang.org/?version=stable&mode=debug&editio...
- LoganDark 2y ago
  Technically trait objects aren't entirely a thing at all. They're a concept that only makes sense in the concept of a pointer (references are safe pointers, `Box`es are smart pointers). You can refer to something as a trait object but the trait of the object is a property of the pointer and not the object. So if you have some struct that implements a trait you can cast a pointer to that struct to a pointer to a trait object, but that struct never stops being the struct, a trait object is just a different way of referring to the struct.
  - iTokio 2y ago
    What about impl Trait then? In that case traits make sense without pointers. To me traits are like a definition of capabilities. A way to duck type things.
    - kaoD 2y ago
      "Trait objects" is lingo for `dyn Trait`. They are a distinct thing from just "trait". They allow virtual dispatch at runtime. See https://doc.rust-lang.org/reference/types/trait-object.html https://doc.rust-lang.org/reference/types/trait-object.html (`dyn Trait`, runtime dynamic dispatch) vs https://doc.rust-lang.org/reference/types/impl-trait.html https://doc.rust-lang.org/reference/types/impl-trait.html (`impl Trait`, compile-time monomorphization)
    - 2y ago
keyle 2y ago
My only experience with Rust has been synchronous mostly with little to show for in terms of async. And I liked Rust. When it ran, I was damn sure it was going to run. There is comfort in that: "Tell me what to fix". The clippy stuff etc. was great too. I read the 3 parts of this website and 'Wowsa'... I'm definitely not going in that direction with Rust. I'll stick to dumb Go code, Swift or Python if I do async heavy stuff. It's hard enough to write good code, I don't see the point of finding walls to smash my head into. Think about it, if you write a lot of async code, chances are you have a ton of latency, waiting on I/O, disk, network etc. Using Rust for this in the first place isn't the wisest since performance isn't as important, most of your performance is wasted 'waiting' on things. Besides Rust wants purity and I/O is gritty little dirt. Sorry my comment turned into a bit of a hot take, feel free to disagree. This async stuff, doesn't look fun at all.
- worik 2y ago
  > My only experience with Rust has been synchronous It is a shame that the dominance of the "async/await" paradigm has made us think in terms of "synchronous" or "async/await" > Think about it, if you write a lot of async code, chances are you have a ton of latency, waiting on I/O, disk, network etc Yes. For almost all code anyone writes blocking code and threads are perfectly OK Asynchronous programming is more trouble, but if trying to deal with a lot of access to those high latency resources asynchronous code really shines. The trouble is that "async/await" is a really bad fit for rust. Every time you say `await` you start invisible magic happening. (A state machine starts spinning I believe in Rust - I may be mistaken) "No invisible magic" was a promise that Rust made to us. What you say is what you mean, and what you mean is what you get. No more, if you use async/await Rust I really do not understand why people who are comfortable with "invisible magic" are not using a language with a garbage collector - that *really* useful invisible magic. Asynchronous programming is the bees knees. It lets you get so much more from your hardware. I learnt to do it implementing telephone switching systems on MS-DOS. We could run seven telephone lines on a 486, with DOS, in (?) about 1991. Async/await has so poisoned the well in Rust that many Rust people do not understand there is more to asynchronous programming than that
  - 2y ago
zavec 2y ago
I really liked the footnote implementation so I checked out his GitHub to see what kind of static site generator he was using, and it looks like he wrote his own? What a baller.
- oconnor663 2y ago
  :-D The styling is adapted from https://edwardtufte.github.io/tufte-css/ https://edwardtufte.github.io/tufte-css/
  - zavec 2y ago
    I am definitely going to have to check that out for my own site!
- thunderseethe 2y ago
  It's also custom logic to embed the code snippets within the article, they're all working code that gets pulled in from rust files. Really stellar stuff.
  - zavec 2y ago
    Yeah! I have a whole laundry list of things I want to improve on my site now
ingen0s 2y ago
Thank you
DeathArrow 2y ago
Async in Rust seems to be modelled after async in .NET.
- vaylian 2y ago
  What similarities are there?
- GolDDranks 2y ago
  I'm not super knowledgeable about async in .NET, but doesn't that have garbage collection as a prerequisite?
  - DeathArrow 2y ago
    No, it's a state machine.
deleted 2y ago
[deleted]

162 points by oconnor663 2y ago

It's amusing that the Rust Playground lets you run a hundred threads. That's generous of them. There's a ceiling below 1000, though. The author points out, "On my Linux laptop I can spawn almost 19k threads before I hit this crash, but the Playground has tighter resource limits." Large servers can go higher than that. The thread-based I/O example with the compute bound poll loop is kind of strange. "Join" isn't really that useful when you have unrelated threads running finite tasks. Usually, you let the thread do its thing,finish, put results on a queue, and let the thread exit. Then it doesn't matter who finishes first. Rust join is actually optional. You don't have to join to close out a thread and reap the thread resources. It's not like zombies in Unix/Linux, where some resources are tied up until the parent process calls wait(). Loops where you join all the threads that are supposedly finished are usually troublesome. If somebody gets stuck, the joiner stalls. Clean exits from programs with lots of channels are troublesome. Channels with multiple senders don't close until all senders exit, which can be hard to arrange when something detects an error. In Rust, the main thread is special. (I consider this unfortunate, but web people like it, because inside browsers, the main thread is very special.) If the main thread exits, all the other threads are silently killed.

diath 2y ago
> Loops where you join all the threads that are supposedly finished are usually troublesome. If somebody gets stuck, the joiner stalls. That makes sense if the main thread is actually doing any useful work, but when its only job is to spawn threads and wait for them to finish before exiting, then it's a pretty common idiom.
willglynn 2y ago
> In Rust, the main thread is special. (I consider this unfortunate, but web people like it, because inside browsers, the main thread is very special.) If the main thread exits, all the other threads are silently killed. Rust inherits this from `pthread_detach()`: The detached attribute merely determines the behavior of the system when the thread terminates; it does not prevent the thread from being terminated if the process terminates using exit(3) (or equivalently, if the main thread returns).
- wahern 2y ago
  The main thread is special because that's how the runtime works on Unix. In particular, when "main" exits, the process exits. This is required by the C standard. It's also fundamentally built into how Unix processes work, as certain global variables, like argv and environ strings, typically are stored on the main thread's stack, so if the main thread is destroyed those references become invalid. In principle Rust could have defined its environment to not make the main thread special, but then it would need some additional runtime magic on Unix systems, including having the main thread poll for all other threads to exit, which in turn would require it to add a layer of indirection to the system's threading runtime (e.g. wrapping pthreads) to be able to track all threads.
  - kelnos 2y ago
    > In principle Rust could have defined its environment to not make the main thread special... Not to mention they'd have to be very careful with what they do on the main thread after they start up the application's first thread (e.g. allocating memory via malloc() is out), since there are quite a few things that are not safe to do (like fork() that's not immediately followed by exec()) in a multi-threaded program. So even a "single-threaded" Rust program would become multi-threaded, and assume all those problems.
oconnor663 2y ago
> Rust join is actually optional. I was recently surprised to learn that returning from main() with background threads still running is more or less UB in C++, because those threads can race against static destructors: https://www.reddit.com/r/cpp/comments/1fu0y6n/when_a_background_thread_races_against_the https://www.reddit.com/r/cpp/comments/1fu0y6n/when_a_backgro.... C doesn't have this issue, though, as far as I know?
- dwattttt 2y ago
  atexit enters the chat
- loeg 2y ago
  Common C implementations (clang, gcc) have static destructors as an extension, though C codebases probably use this a lot less than C++ ones with static objects and destructor methods.
shepmaster 2y ago
> the Rust Playground lets you run a hundred threads It's more that we don't do anything to prevent it, other than coarse process-wide memory / CPU time limits. IIRC, Rust-spawned threads on Linux use 2MiB of stack space by default, so that seems like a likely cap. Note that the playground is only 2 cores and you are sharing with everyone else, so you aren't likely to really benefit.
- littlestymaar 2y ago
  > Note that the playground is only 2 cores and you are sharing with everyone else This is amazing, I use it all the time with no performance issues so I expected it to be much beefier to support many simultaneous users. How many users does it serve? (Monthly or daily user and/or compilation job sent). And what tricks are used to keep it working? (I suspect it can re-use already compiled binaries of all supported dependencies and only need to compile the user's code and link it, but is there other clever strategies?)
  - shepmaster 2y ago
    > How many users does it serve? I don't really track users, but over the last 24 hours, there were 47.8k meaningful [1] requests taking a total of 28.2 hours. That ~0.5 requests per second number has been relatively consistent. > re-use already compiled binaries of all supported dependencies and only need to compile the user's code and link it, but is there other clever strategies? Yes, we pre-compile all the available dependencies [2] and that's about it. > I use it all the time with no performance issues That's good to hear! There's been a long-running bug where the playground binary loses track of the child Docker container (maybe?) and then the machine runs out of memory and the OOM killer often does more harm than good [3]. While trying to pin that down, I've recently caused the entire process to get into what appears to be a complete deadlock where no requests can be serviced at all. This tends to happen while I'm asleep so either I have no chance to debug it before it is auto-killed or the playground is unresponsive for 8+ hours. [1]: compiling / executing code, running clippy/miri/rustfmt, expanding macros [2]: https://github.com/rust-lang/rust-playground/blob/c4d00b90aab3742c841011b29fba038f3cd99690/compiler/base/Dockerfile#L105-L107 https://github.com/rust-lang/rust-playground/blob/c4d00b90aa... [3]: somehow it does something that ends up killing the network stack and then the machine is basically dead in the water. Very similar to what is reported in https://serverfault.com/q/1125634/119136 https://serverfault.com/q/1125634/119136
dwattttt 2y ago
> Loops where you join all the threads that are supposedly finished are usually troublesome. If somebody gets stuck, the joiner stalls. Clean exits from programs with lots of channels are troublesome. Channels with multiple senders don't close until all senders exit, which can be hard to arrange when something detects an error. I wish join-with-timeout was a more common/supported operation.

alilleybrinker 2y ago

In part two, the author explains trait objects in a way that is, I think, a little misleading. They're right that trait objects are dynamically sized types, which means they can't be passed by value to functions, but wrong that they need to be boxed; they can instead be put behind a reference. Both of the following are valid types. type DynFutureBox = Pin<Box<dyn Future<Output = ()>>>; type DynFutureRef<'f> = Pin<&'f dyn Future<Output = ()>>; You can see this in the Rust Playground here: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=6c61667f75dfc2435e6a342f897d1c30 https://play.rust-lang.org/?version=stable&mode=debug&editio...

LoganDark 2y ago
Technically trait objects aren't entirely a thing at all. They're a concept that only makes sense in the concept of a pointer (references are safe pointers, `Box`es are smart pointers). You can refer to something as a trait object but the trait of the object is a property of the pointer and not the object. So if you have some struct that implements a trait you can cast a pointer to that struct to a pointer to a trait object, but that struct never stops being the struct, a trait object is just a different way of referring to the struct.
- iTokio 2y ago
  What about impl Trait then? In that case traits make sense without pointers. To me traits are like a definition of capabilities. A way to duck type things.
  - kaoD 2y ago
    "Trait objects" is lingo for `dyn Trait`. They are a distinct thing from just "trait". They allow virtual dispatch at runtime. See https://doc.rust-lang.org/reference/types/trait-object.html https://doc.rust-lang.org/reference/types/trait-object.html (`dyn Trait`, runtime dynamic dispatch) vs https://doc.rust-lang.org/reference/types/impl-trait.html https://doc.rust-lang.org/reference/types/impl-trait.html (`impl Trait`, compile-time monomorphization)
  - 2y ago

keyle 2y ago

My only experience with Rust has been synchronous mostly with little to show for in terms of async. And I liked Rust. When it ran, I was damn sure it was going to run. There is comfort in that: "Tell me what to fix". The clippy stuff etc. was great too. I read the 3 parts of this website and 'Wowsa'... I'm definitely not going in that direction with Rust. I'll stick to dumb Go code, Swift or Python if I do async heavy stuff. It's hard enough to write good code, I don't see the point of finding walls to smash my head into. Think about it, if you write a lot of async code, chances are you have a ton of latency, waiting on I/O, disk, network etc. Using Rust for this in the first place isn't the wisest since performance isn't as important, most of your performance is wasted 'waiting' on things. Besides Rust wants purity and I/O is gritty little dirt. Sorry my comment turned into a bit of a hot take, feel free to disagree. This async stuff, doesn't look fun at all.

worik 2y ago
> My only experience with Rust has been synchronous It is a shame that the dominance of the "async/await" paradigm has made us think in terms of "synchronous" or "async/await" > Think about it, if you write a lot of async code, chances are you have a ton of latency, waiting on I/O, disk, network etc Yes. For almost all code anyone writes blocking code and threads are perfectly OK Asynchronous programming is more trouble, but if trying to deal with a lot of access to those high latency resources asynchronous code really shines. The trouble is that "async/await" is a really bad fit for rust. Every time you say `await` you start invisible magic happening. (A state machine starts spinning I believe in Rust - I may be mistaken) "No invisible magic" was a promise that Rust made to us. What you say is what you mean, and what you mean is what you get. No more, if you use async/await Rust I really do not understand why people who are comfortable with "invisible magic" are not using a language with a garbage collector - that *really* useful invisible magic. Asynchronous programming is the bees knees. It lets you get so much more from your hardware. I learnt to do it implementing telephone switching systems on MS-DOS. We could run seven telephone lines on a 486, with DOS, in (?) about 1991. Async/await has so poisoned the well in Rust that many Rust people do not understand there is more to asynchronous programming than that
- 2y ago

zavec 2y ago

I really liked the footnote implementation so I checked out his GitHub to see what kind of static site generator he was using, and it looks like he wrote his own? What a baller.

oconnor663 2y ago
:-D The styling is adapted from https://edwardtufte.github.io/tufte-css/ https://edwardtufte.github.io/tufte-css/
- zavec 2y ago
  I am definitely going to have to check that out for my own site!
thunderseethe 2y ago
It's also custom logic to embed the code snippets within the article, they're all working code that gets pulled in from rust files. Really stellar stuff.
- zavec 2y ago
  Yeah! I have a whole laundry list of things I want to improve on my site now

ingen0s 2y ago

Thank you

DeathArrow 2y ago

Async in Rust seems to be modelled after async in .NET.

vaylian 2y ago
What similarities are there?
GolDDranks 2y ago
I'm not super knowledgeable about async in .NET, but doesn't that have garbage collection as a prerequisite?
- DeathArrow 2y ago
  No, it's a state machine.

deleted 2y ago

[deleted]