8 ms·

Green threads explained in 200 lines of Rust

374 points by taheris 7y ago

joaomoreno 7y ago
Beautiful docs
bluejekyll 7y ago
The explanation of the asm! macro is great, I’ve always been a little confused by it. It does make me thinking about how this could be useful in the context of Futures and async/await.
- steveklabnik 7y ago
  > It does make me thinking about how this could be useful in the context of Futures and async/await. At its core, when you pass a Future (really, a chain of futures, but in the end that's still a Future) to an executor, the executor creates a stack for it. An executor with multiple futures will then switch between them, sorta similar to the code seen here. The details depend on the exact executor, of course, but the fundamental idea is the same. One of the nice things about the futures design is that you can know exactly how big the stack needs to be, so the resizing stuff isn't an issue.
  - cfsamson 7y ago
    The long term plan for this was/is to actually connect them to futures and the async story in rust, adapting this to be an executor and implementing our own futures (though I probably have to implement a simple reactor and fake some IO operation in e seperate thread as well). However, it's quite a bit of rework and better to seperate this in two parts, but I feel it could be a good way to build brick by brick towards a pretty good understanding for those interested.
amelius 7y ago
Isn't one downside that the inline assembly will not be peephole-optimized by the code generator (LLVM)? E.g., you'd be saving and restoring registers that are not even used.
- snissn 7y ago
  > a simple but working example
  - amelius 7y ago
    Yeah, of course, but how would you work this type of code into something that can be used in production? The goal should be to have a green thread library that you can just use without thinking about registers or assembly.
    - heyzooos 7y ago
      > I’m not trying to make a perfect implementation here. I’m cutting corners to get down to the essence and fit it into what was originally intended to be an article but expanded into a small book. lol
    - arnarbi 7y ago
      > how would you work this type of code into something that can be used in production? You wouldn't. There are production-ready solutions to this in many languages. > The goal should be to have a green thread library that you can just use without thinking about registers or assembly. The goal here is very explicitly to explain a concept by example, which is largely incompatible with what you say the goal should be. I think the author did a very good job.
      - amelius 7y ago
        > I think the author did a very good job. I wasn't questioning that!
    - masklinn 7y ago
      > Yeah, of course, but how would you work this type of code into something that can be used in production? Why would you? > The goal should be to have a green thread library that you can just use without thinking about registers or assembly. No it should not. That's a completely separate (and mostly incompatible) goal than TFA's, which is educational.
FullyFunctional 7y ago
Co-routines are very useful and likely underused, but sometimes you are actually better off being able to pass the control to a given thread directly, other than having a scheduler involved. Anecdote: almost a decade ago, I was responsible for an NVMe-like implementation (hard- and software). The 3rd version of the firmware recognized the various components as threads, but there was no need for preemption (which would require expensive locking). Traditional scheduling would work, but you actually know exactly which thread should execute next (hardware will signal done), so an explicit yield_to() was far cheaper and only slightly more expensive than a function call.
- PopeDotNinja 7y ago
  If you have any knowledge of it, what are your thoughts about the Erlang scheduler & Erlang's concurrency model?
  - FullyFunctional 7y ago
    Funny enough we actually started prototyping with Erlang for subsequent project (which was cancelled before it went far). Unfortunately I don't know enough to know what's special about the Erlang scheduler (if anything), but as I understand the Erlang concurrency model, it's mostly about not sharing memory (forces explicit communication). That's obviously going to eliminate a host of bugs, but it would have been way too expensive for the mentioned firmware. Once we got started with Erlang I was pretty turned off. The pretty examples you see in tutorials aren't what you'll be using. Instead it's framework upon framework, far from elegance IMO. I was happy to not have to deal with that again. Today I'd probably choose Rust for the same task (static types FTW). (EDIT: typos)
    - PopeDotNinja 7y ago
      Given that Erlang is sufficiently different from anything else I've seen, it doesn't surprise me that trying to be productive in Erlang before you knew it well enough was suboptimal experience. I like it, and more specifically Elixir, quite a bit, but the learning curve was steep.
amluto 7y ago
Part of the code is: unsafe { std::ptr::write(stack_ptr.offset(SSIZE - 16) as *mut u64, hello as u64); ctx.rsp = stack_ptr.offset(SSIZE - 16) as u64; gt_switch(&mut ctx, &mut n); } Only the last line should be unsafe — the first line appears to be writing in bounds to a Vec, which is easy (and much more readable) in safe Rust. Linux used to do its actual context switch in C with inline asm, and it changed to being in straight asm a while back. This was absolutely a win. Rather than trying to make everything work out behind the compiler’s back, it’s much more straightforward to write a function in asm that does the stack switch.
- gravitas 7y ago
  Fantastic (IMHO) read on the last paragraph's history and code movement: https://www.maizure.org/projects/evolution_x86_context_switch_linux/ https://www.maizure.org/projects/evolution_x86_context_switc...
- oconnor663 7y ago
  The ptr::write is needed because the stack vector contains bytes and the code wants to do a pointer cast and write a u64. It could be done in safe Rust with u64::to_ne_bytes and a copy_from_slice or something like that, but since all of this code is extravagantly unsafe anyway, I think it's reasonably clear to Just Do It :)
- paines 7y ago
  >Linux used to do its actual context switch in C with inline asm, and it changed to being in straight asm a while back. Could you be a bit more specific, maybe a link to the diff e.g. THANKS!
fc_barnes 7y ago
Daily reminder that goroutines don't require a context switch.
- steveklabnik 7y ago
  A kernel context switch. Neither do the ones described here.
- deleted 7y ago
  [deleted]
mhh__ 7y ago
https://github.com/dlang/druntime/blob/v2.086.0/src/core/thread.d#L4071 https://github.com/dlang/druntime/blob/v2.086.0/src/core/thr... D's implementation ^ (Linked partly because the inline assembly is very clear)
swolchok 7y ago
FWIW, there is POSIX-specified functionality (swapcontext and friends) for changing the user thread context: http://pubs.opengroup.org/onlinepubs/009695399/functions/swapcontext.html http://pubs.opengroup.org/onlinepubs/009695399/functions/swa...
- asdfasgasdgasdg 7y ago
  It's very slow though, at least if you believe the Boost docs. They have various context-controlling types. One of them is called ucontext_t, and falls back to ucontext. Then there's fcontext_t, which is basically a more advanced version of this article. Boost's docs claim it is much faster than ucontext.
kccqzy 7y ago
Great explanation! Although I have to wonder, why did the author choose to use Rust for this project? As far as I can see, it didn't really use any of Rust's advanced features not available in C or C++, like tagged unions, pattern matching, advanced type system, traits, borrow checking… For what it's worth, I translated the whole code into C++. And it's essentially the same: https://gist.github.com/kccqzy/c404b8614f39f854f137dcc9284b0b87 https://gist.github.com/kccqzy/c404b8614f39f854f137dcc9284b0... And it runs correctly on macOS and Linux when compiled with clang.
- kasane 7y ago
  Why would one pick C or C++ over Rust for this project?
  - sim_card_map 7y ago
    Much easier to build, for me at least. I tried installing Rust and building the example, but then it complained about nightly features, and I didn't have time to figure that out. C/C++ are mainstream languages, and building is as simple as g++ test.cpp.
    - eridius 7y ago
      Using nightly features is as simple as inserting +nightly as the first argument to rustc or cargo (assuming rust was installed with rustup, which is the default way to install it). So in this case, it's just `rustc +nightly green_threads.rs`
  - kccqzy 7y ago
    If the point is to explain how green threads work, it is better to use a common language so people can simply try to understand the concept and not try to understand both the concept and the language at the same time.
- sim_card_map 7y ago
  Thanks a lot for this!
identity0 7y ago
I feel like "200 lines of Rust" is a little misleading considering how much explaination there is. It's like saying you can learn all of string theory in 0 lines of Python.
- mlevental 7y ago
  it seems like in every post there's a naysayer about something. you're complaining about an article that explains a difficult topic extensively and generously - in my day you had to pay really really good money for that. who cares if the title focuses one aspect of the explanation? sure it's a little deceptive but it's such a minor cost to you that you literally spent more time complaining than they deceived you out (click link with false notion, see length, close page). LPT: being critical in and of itself does not make you look smart.
deleted 7y ago
[deleted]
iamcreasy 7y ago
Now that there is a lot more interest in green threads, I was wondering if anybody knows why back in the days Java dropped the support of green thread in favor of native thread?

374 points by taheris 7y ago

Beautiful docs

The explanation of the asm! macro is great, I’ve always been a little confused by it. It does make me thinking about how this could be useful in the context of Futures and async/await.

steveklabnik 7y ago
> It does make me thinking about how this could be useful in the context of Futures and async/await. At its core, when you pass a Future (really, a chain of futures, but in the end that's still a Future) to an executor, the executor creates a stack for it. An executor with multiple futures will then switch between them, sorta similar to the code seen here. The details depend on the exact executor, of course, but the fundamental idea is the same. One of the nice things about the futures design is that you can know exactly how big the stack needs to be, so the resizing stuff isn't an issue.
- cfsamson 7y ago
  The long term plan for this was/is to actually connect them to futures and the async story in rust, adapting this to be an executor and implementing our own futures (though I probably have to implement a simple reactor and fake some IO operation in e seperate thread as well). However, it's quite a bit of rework and better to seperate this in two parts, but I feel it could be a good way to build brick by brick towards a pretty good understanding for those interested.

amelius 7y ago

Isn't one downside that the inline assembly will not be peephole-optimized by the code generator (LLVM)? E.g., you'd be saving and restoring registers that are not even used.

snissn 7y ago
> a simple but working example
- amelius 7y ago
  Yeah, of course, but how would you work this type of code into something that can be used in production? The goal should be to have a green thread library that you can just use without thinking about registers or assembly.
  - heyzooos 7y ago
    > I’m not trying to make a perfect implementation here. I’m cutting corners to get down to the essence and fit it into what was originally intended to be an article but expanded into a small book. lol
  - arnarbi 7y ago
    > how would you work this type of code into something that can be used in production? You wouldn't. There are production-ready solutions to this in many languages. > The goal should be to have a green thread library that you can just use without thinking about registers or assembly. The goal here is very explicitly to explain a concept by example, which is largely incompatible with what you say the goal should be. I think the author did a very good job.
    - amelius 7y ago
      > I think the author did a very good job. I wasn't questioning that!
  - masklinn 7y ago
    > Yeah, of course, but how would you work this type of code into something that can be used in production? Why would you? > The goal should be to have a green thread library that you can just use without thinking about registers or assembly. No it should not. That's a completely separate (and mostly incompatible) goal than TFA's, which is educational.

FullyFunctional 7y ago

Co-routines are very useful and likely underused, but sometimes you are actually better off being able to pass the control to a given thread directly, other than having a scheduler involved. Anecdote: almost a decade ago, I was responsible for an NVMe-like implementation (hard- and software). The 3rd version of the firmware recognized the various components as threads, but there was no need for preemption (which would require expensive locking). Traditional scheduling would work, but you actually know exactly which thread should execute next (hardware will signal done), so an explicit yield_to() was far cheaper and only slightly more expensive than a function call.

PopeDotNinja 7y ago
If you have any knowledge of it, what are your thoughts about the Erlang scheduler & Erlang's concurrency model?
- FullyFunctional 7y ago
  Funny enough we actually started prototyping with Erlang for subsequent project (which was cancelled before it went far). Unfortunately I don't know enough to know what's special about the Erlang scheduler (if anything), but as I understand the Erlang concurrency model, it's mostly about not sharing memory (forces explicit communication). That's obviously going to eliminate a host of bugs, but it would have been way too expensive for the mentioned firmware. Once we got started with Erlang I was pretty turned off. The pretty examples you see in tutorials aren't what you'll be using. Instead it's framework upon framework, far from elegance IMO. I was happy to not have to deal with that again. Today I'd probably choose Rust for the same task (static types FTW). (EDIT: typos)
  - PopeDotNinja 7y ago
    Given that Erlang is sufficiently different from anything else I've seen, it doesn't surprise me that trying to be productive in Erlang before you knew it well enough was suboptimal experience. I like it, and more specifically Elixir, quite a bit, but the learning curve was steep.

amluto 7y ago

Part of the code is: unsafe { std::ptr::write(stack_ptr.offset(SSIZE - 16) as *mut u64, hello as u64); ctx.rsp = stack_ptr.offset(SSIZE - 16) as u64; gt_switch(&mut ctx, &mut n); } Only the last line should be unsafe — the first line appears to be writing in bounds to a Vec, which is easy (and much more readable) in safe Rust. Linux used to do its actual context switch in C with inline asm, and it changed to being in straight asm a while back. This was absolutely a win. Rather than trying to make everything work out behind the compiler’s back, it’s much more straightforward to write a function in asm that does the stack switch.

gravitas 7y ago
Fantastic (IMHO) read on the last paragraph's history and code movement: https://www.maizure.org/projects/evolution_x86_context_switch_linux/ https://www.maizure.org/projects/evolution_x86_context_switc...
oconnor663 7y ago
The ptr::write is needed because the stack vector contains bytes and the code wants to do a pointer cast and write a u64. It could be done in safe Rust with u64::to_ne_bytes and a copy_from_slice or something like that, but since all of this code is extravagantly unsafe anyway, I think it's reasonably clear to Just Do It :)
paines 7y ago
>Linux used to do its actual context switch in C with inline asm, and it changed to being in straight asm a while back. Could you be a bit more specific, maybe a link to the diff e.g. THANKS!

fc_barnes 7y ago

Daily reminder that goroutines don't require a context switch.

steveklabnik 7y ago
A kernel context switch. Neither do the ones described here.
deleted 7y ago
[deleted]

mhh__ 7y ago

https://github.com/dlang/druntime/blob/v2.086.0/src/core/thread.d#L4071 https://github.com/dlang/druntime/blob/v2.086.0/src/core/thr... D's implementation ^ (Linked partly because the inline assembly is very clear)

swolchok 7y ago

FWIW, there is POSIX-specified functionality (swapcontext and friends) for changing the user thread context: http://pubs.opengroup.org/onlinepubs/009695399/functions/swapcontext.html http://pubs.opengroup.org/onlinepubs/009695399/functions/swa...

asdfasgasdgasdg 7y ago
It's very slow though, at least if you believe the Boost docs. They have various context-controlling types. One of them is called ucontext_t, and falls back to ucontext. Then there's fcontext_t, which is basically a more advanced version of this article. Boost's docs claim it is much faster than ucontext.

kccqzy 7y ago

Great explanation! Although I have to wonder, why did the author choose to use Rust for this project? As far as I can see, it didn't really use any of Rust's advanced features not available in C or C++, like tagged unions, pattern matching, advanced type system, traits, borrow checking… For what it's worth, I translated the whole code into C++. And it's essentially the same: https://gist.github.com/kccqzy/c404b8614f39f854f137dcc9284b0b87 https://gist.github.com/kccqzy/c404b8614f39f854f137dcc9284b0... And it runs correctly on macOS and Linux when compiled with clang.

kasane 7y ago
Why would one pick C or C++ over Rust for this project?
- sim_card_map 7y ago
  Much easier to build, for me at least. I tried installing Rust and building the example, but then it complained about nightly features, and I didn't have time to figure that out. C/C++ are mainstream languages, and building is as simple as g++ test.cpp.
  - eridius 7y ago
    Using nightly features is as simple as inserting +nightly as the first argument to rustc or cargo (assuming rust was installed with rustup, which is the default way to install it). So in this case, it's just `rustc +nightly green_threads.rs`
- kccqzy 7y ago
  If the point is to explain how green threads work, it is better to use a common language so people can simply try to understand the concept and not try to understand both the concept and the language at the same time.
sim_card_map 7y ago
Thanks a lot for this!

identity0 7y ago

I feel like "200 lines of Rust" is a little misleading considering how much explaination there is. It's like saying you can learn all of string theory in 0 lines of Python.

mlevental 7y ago
it seems like in every post there's a naysayer about something. you're complaining about an article that explains a difficult topic extensively and generously - in my day you had to pay really really good money for that. who cares if the title focuses one aspect of the explanation? sure it's a little deceptive but it's such a minor cost to you that you literally spent more time complaining than they deceived you out (click link with false notion, see length, close page). LPT: being critical in and of itself does not make you look smart.

deleted 7y ago

[deleted]

iamcreasy 7y ago

Now that there is a lot more interest in green threads, I was wondering if anybody knows why back in the days Java dropped the support of green thread in favor of native thread?