22 ms·

Async2 – The .NET Runtime Async experiment concludes

137 points by Nelkins 2y ago

xeromal 2y ago
It's a bummer green threads didn't work out
giancarlostoro 2y ago
I still remember first hearing about Rust and how it was using Green Threads, then out of the blue a year later passes or months, and I'm reading that it doesn't do any of that anymore, it's basically C++ on steroids mixed with functional programming. I never did look up why they gave up on Green threads and other things, I wonder if they faced similar challenges?
- wyager 2y ago
  Rust has settled into an excellent design niche where the language does not provide any features that require a language runtime. Rust's async/await is a purely compile-time feature, with no runtime support required. This is extremely powerful because it means you can do things like easily run async code on embedded devices with no OS. To run async code in rust, you bring your own executor. The most popular one is tokio (for desktop OSs), but you also have stuff like pollster (for running async code as blocking code) and embassy or RTIC (for embedded).
  - jtrueb 2y ago
    Agreed, as much as there are complaints about usability of async in Rust, many alternatives proposed would fail in resource constrained systems. Green threading would be impossible or very limiting on embedded, where each stack eats away at something like 1-2% of memory and allocation is likely impossible or prohibitively expensive. I do note that some of the futures are starting to take up a couple KiB of RAM and ROM.
  - eknkc 2y ago
    I might be completely wrong as I only played with rust but the current method seems to couple any async libraries you’d like to write to the executor. Which feels wrong imo. Would it be feasible to provide async io (file, network) code in std and let the executors execute? Or even some std traits that abstract the filesystem, network etc so that the executors can bring their own implementations but at least library code could only depend on std?
    - wyager 2y ago
      Correct, libraries do have to either pick a specific async IO library or use a trait for async IO. I have seen some of the latter in embedded-land, although I'm not sure if it's used in desktop async rust as well. The rust compiler currently discourages async traits with a warning, which seems wrong to me, but we use them extensively in embedded anyway.
      - anonymoushn 2y ago
        The feature is less than 1 year old, so a lot of the ecosystem couldn't use the feature because of being written before the feature existed. It does seem silly to have a warning today.
  - anonymoushn 2y ago
    This is a fine approach but it's trivial to ship green threads with no runtime support as well. Various libraries implementing the feature exist for C or Zig etc.
    - kbolino 2y ago
      If you can accept cooperative scheduling that runs tasks only on a single core, then yes, lightweight solutions exist. This is more like "event-driven coroutines" than "green threads" though. If you want "full" green threads with preemptive scheduling and parallel execution across multiple cores, then an active runtime is needed.
- dboreham 2y ago
  User level (aka green) threading is difficult. Very few implementations work (Erlang, Golang, <any others>+) and it takes a great deal of time and effort to get to that "fully working" state. +Perhaps the new Java fiber stuff works but I don't have enough data to be sure yet.
- Ygg2 2y ago
  There were plans for green threads, however, green threads were abandoned due to runtime costs around like 6 years ago? https://github.com/rust-lang/rfcs/blob/master/text/0230-remove-runtime.md https://github.com/rust-lang/rfcs/blob/master/text/0230-remo... EDIT: 10 years ago.
  - steveklabnik 2y ago
    It’s coming up on ten years. Time flies.
    - Ygg2 2y ago
      I thought it was 2018, it was 2014 instead.
  - anonymoushn 2y ago
    "green threads" as discussed in TFA don't require a runtime. The relevant issues are maybe https://github.com/rust-lang/rust/issues/33368 https://github.com/rust-lang/rust/issues/33368 and https://internals.rust-lang.org/t/what-is-the-current-safety-story-for-library-based-stackful-coroutines/14863/16 https://internals.rust-lang.org/t/what-is-the-current-safety...
- steveklabnik 2y ago
  https://github.com/rust-lang/rfcs/blob/master/text/0230-remove-runtime.md https://github.com/rust-lang/rfcs/blob/master/text/0230-remo... Note this was in 2014, before Rust 1.0.
  - hyutmyjahsd 2y ago
    I miss peak rust.
  - giancarlostoro 2y ago
    Yes it was a while back, wow 10 years.
- jasode 2y ago
  >I still remember first hearing about Rust and how it was using Green Threads, [...] I never did look up why they gave up on Green threads It's because green threads have unavoidable performance costs. I collected various threads that has Rust contributors explaining the costs they didn't want to pay: https://news.ycombinator.com/item?id=39062953 https://news.ycombinator.com/item?id=39062953 A language like Go is willing to pay those performance costs because they deliberately sit at a higher level of abstraction than lower-level Rust. Each chose different tradeoffs for different goals: - Golang : green threads worth the perf cost to gain higher productivity (goroutines, channels, etc) for the type of programs that Go programmers like to write - Rust : green threads are not worth the perf cost so as to not sacrifice the best possible performance for the type of programs Rust programmers like to write You can't have a language+runtime that satisfies both camps 100% perfectly. The language designer must choose the tradeoff. But because this tradeoff decision is not blatantly well-known and disseminated ... it also perpetually confuses Language X proponents on why Language Y didn't do the same thing as Language X.
  - 01HNNWZ0MV43FF 2y ago
    What do Go channels do that takes advantage of their green threads?
  - giancarlostoro 2y ago
    Thank you for this! This is exactly why I posted my comment, I was hoping someone would share some insight about Rust.
- zamalek 2y ago
  Rust can have green threads. It depends on the reactor. The abstraction sits one level lower than nearly everything else out there.
cube2222 2y ago
This all makes sense but > Green threads are different. The memory of a green thread is allocated on the heap. But all of this comes with a cost: As they aren't managed by the OS, they can't take advantage of multiple cores inherently. But for I/O-bound operations, they are a good fit. this is clearly not true? Am I missing some nuance here, as I'm sure the author knows what they're talking about? Green threads can totally use a multi-threaded runtime, like e.g. Go does, and it works just fine. The main hurdle with them is arguably FFI.
- DougBTX 2y ago
  The “inherently” means not by default, i.e., the runtime has to support moving green threads between OS threads itself.
  - cube2222 2y ago
    Ah, that makes sense, thanks!
  - 01HNNWZ0MV43FF 2y ago
    That's not how I use "inherent", maybe they should just say "default" then? But that's like... C is also single threaded by default, what isn't?
    - jayd16 2y ago
      They will never be transparently/fundamentalally managed by the OS alone. The runtime will need to determine how to juggle green threads across multiple OS threads. In that way, this mapping is not inherent. It can be designed around but that itself is a runtime design decision and I would not say it's akin to default vs custom.
    - recursive 2y ago
      With respect, it's not particularly relevant how you use "inherent". It's a standard usage. Rather than asking the whole rest of the world to change, you should probably learn the definition.
bob1029 2y ago
I don't know if this proposal makes a lot of sense. The existing async1/TPL path is stable & predictable. If you find yourself needing more performance, you can reach for hardware Thread instances and use whatever locking/waiting/sharing/context strategies you desire. Anything else is a weird blend of compromises that is going to have caveats that are not immediately obvious for your specific use case. For example, async2 w/ runtime JIT appears to have some tradeoffs with regard to GC & memory usage and the experiment writeup leaves some open-ended questions here[0]. [0]: https://github.com/dotnet/runtimelab/blob/feature/async2-experiment/docs/design/features/runtime-handled-tasks.md#impact-on-gc-heap-size-pauses https://github.com/dotnet/runtimelab/blob/feature/async2-exp...
- whaleofatw2022 2y ago
  Trying to do clever stuff with threads tends to lead from brow beating, wailing, and gnashing of teeth by various folks who insist to just trust the threadpool.
- GordonS 2y ago
  > If you find yourself needing more performance, you can reach for hardware Thread instances True for code your team write, but async/await is kind of viral, and many libraries now only have async APIs, which makes them difficult to shoehorn into a threading approach.
  - bob1029 2y ago
    > async/await is kind of viral Exactly part of my concern. Hypothetically we've got 2 competing viruses now.
    - GordonS 2y ago
      Surely it's the same "virus"? Unless I'm mistaken, the async2 proposal wouldn't require any code changes, as it essentialy just moves compiler stuff to the runtime.
      - 2y ago
neonsunset 2y ago
For everyone reading this blog post I caution to take the conclusions there with a grain of salt as they are an interpretation of the notes written down here: https://github.com/dotnet/runtimelab/blob/feature/async2-experiment/docs/design/features/runtime-handled-tasks.md https://github.com/dotnet/runtimelab/blob/feature/async2-exp... It is difficult to draw conclusions at the present moment on e.g. memory consumption until the work on this, which is underway, makes it into mainline runtime. It's important to understand that the experiment was first and foremost a research to look into modernizing async implementation, and was a massive success. Now once that is proven, the tuned and polished implementation will be made. Once it is done and makes into a release (it could even be as early as .NET 10), then further review will be possible. With that said, thank you for writing it, .NET tends to be criminally underrated and slept on by the most of the industry, and these help to alleviate it even if just a bit.
- hirvi74 2y ago
  > .NET tends to be criminally underrated and slept on by the most of the industry I have been programming in .Net/C# for about 8 years now. I absolutely hate Microsoft with every fiber of my being. However, I can't thank them enough for C# (and all the FOSS contributors as well). .Net has been such a pleasant experience that I truly do not want to program in any other language.
  - naasking 2y ago
    > I absolutely hate Microsoft with every fiber of my being. Why? I get it back when they were super hostile to Linux and open source, but those days are long past.
    - anonymoushn 2y ago
      Maybe because GP uses Windows sometimes.
    - orphea 2y ago
      They are super hostile to users of their OS, and these days are right now.
ikekkdcjkfke 2y ago
What is the fundemental problem we are trying to solve here? My pet problem with async is that it takes so much syntax, and i wonder why we need to do that. As i understand it it is all about blocking calls taking up OS threads. So i will try to naievly solve it here, and maybe i end up with the same conclusions. When doing a blocking call, the OS thread could just start executing something else transparently and not 'yield' anything to the code, it just stops executing the code and comes back and executes further, no await, no tasks no nothing. Ok, but how did we end up in a thread? The http listener started 4 threads and it's just putting a stack of function pointers and context memory for the threads to eat when they want. There is a separate engine on another OS thread that handles where the threads can start executing ready code again. No Task or await keywords show up in this code. I have no idea how stack traces work but i guess that can just be saved to memory and loaded back again when the threads feel like executing some ready code
- irdc 2y ago
  The rationale for async/await I keep hearing is that the separate stack required for each thread uses a lot of memory. I’m not sure going full on function colouring is the answer though. A concerted effort for language runtimes to use less stack space and then simply allocating smaller thread stacks sounds to me like a more elegant solution. It certainly is a whole lot easier to debug than a deeply-nested async/await chain.
  - iknowstuff 2y ago
    But also avoiding the cost of context switching, and the ability to handle many tasks in embedded environments. https://tweedegolf.nl/en/blog/65/async-rust-vs-rtos-showdown https://tweedegolf.nl/en/blog/65/async-rust-vs-rtos-showdown
    - irdc 2y ago
      This makes it sound like using async/await is more like hand-optimising your code, somewhat similar to rewriting it in assembler. If a way can be found to improve thread switching times (eg. by only switching registers the program actually uses), the fact that the threaded code is easier to debug and reason about becomes a large advantage.
kodablah 2y ago
> And for the transition phase, there has to be interop for async ↔ async2 For those like me who weren't clear whether `async2` was expected to be a real keyword in the final language, it's not[0]. 0 - https://github.com/dotnet/runtime/issues/94620#issuecomment-1883524868 https://github.com/dotnet/runtime/issues/94620#issuecomment-...
- augusto-moura 2y ago
  async2 looks like such a terrible keyword name, that it didn't even crossed my mind as an option. Only following your linked comment I understood that they used it as a temporary name while testing
- TwentyPosts 2y ago
  Have you read the article? > As async is already used as an identifier in C#, the team decided to use async2 as a codename for the experiment. If that thing ever makes it into the runtime, it will be called async - so it will be a replacement for the current async implementation.
jayd16 2y ago
I'm encouraged by the talk of improved exception handling and a recognition of how much garbage it creates. It makes it hard to do things in an idiomatic way and keep GC pressure low. I hope they also revisit task state tracking and TaskCanceledException. It feels out of place to use an exception (and unroll the stack) as control flow in the same API they added ValueTask to keep Task handles on the stack.

137 points by Nelkins 2y ago

It's a bummer green threads didn't work out

I still remember first hearing about Rust and how it was using Green Threads, then out of the blue a year later passes or months, and I'm reading that it doesn't do any of that anymore, it's basically C++ on steroids mixed with functional programming. I never did look up why they gave up on Green threads and other things, I wonder if they faced similar challenges?

wyager 2y ago
Rust has settled into an excellent design niche where the language does not provide any features that require a language runtime. Rust's async/await is a purely compile-time feature, with no runtime support required. This is extremely powerful because it means you can do things like easily run async code on embedded devices with no OS. To run async code in rust, you bring your own executor. The most popular one is tokio (for desktop OSs), but you also have stuff like pollster (for running async code as blocking code) and embassy or RTIC (for embedded).
- jtrueb 2y ago
  Agreed, as much as there are complaints about usability of async in Rust, many alternatives proposed would fail in resource constrained systems. Green threading would be impossible or very limiting on embedded, where each stack eats away at something like 1-2% of memory and allocation is likely impossible or prohibitively expensive. I do note that some of the futures are starting to take up a couple KiB of RAM and ROM.
- eknkc 2y ago
  I might be completely wrong as I only played with rust but the current method seems to couple any async libraries you’d like to write to the executor. Which feels wrong imo. Would it be feasible to provide async io (file, network) code in std and let the executors execute? Or even some std traits that abstract the filesystem, network etc so that the executors can bring their own implementations but at least library code could only depend on std?
  - wyager 2y ago
    Correct, libraries do have to either pick a specific async IO library or use a trait for async IO. I have seen some of the latter in embedded-land, although I'm not sure if it's used in desktop async rust as well. The rust compiler currently discourages async traits with a warning, which seems wrong to me, but we use them extensively in embedded anyway.
    - anonymoushn 2y ago
      The feature is less than 1 year old, so a lot of the ecosystem couldn't use the feature because of being written before the feature existed. It does seem silly to have a warning today.
- anonymoushn 2y ago
  This is a fine approach but it's trivial to ship green threads with no runtime support as well. Various libraries implementing the feature exist for C or Zig etc.
  - kbolino 2y ago
    If you can accept cooperative scheduling that runs tasks only on a single core, then yes, lightweight solutions exist. This is more like "event-driven coroutines" than "green threads" though. If you want "full" green threads with preemptive scheduling and parallel execution across multiple cores, then an active runtime is needed.
dboreham 2y ago
User level (aka green) threading is difficult. Very few implementations work (Erlang, Golang, <any others>+) and it takes a great deal of time and effort to get to that "fully working" state. +Perhaps the new Java fiber stuff works but I don't have enough data to be sure yet.
Ygg2 2y ago
There were plans for green threads, however, green threads were abandoned due to runtime costs around like 6 years ago? https://github.com/rust-lang/rfcs/blob/master/text/0230-remove-runtime.md https://github.com/rust-lang/rfcs/blob/master/text/0230-remo... EDIT: 10 years ago.
- steveklabnik 2y ago
  It’s coming up on ten years. Time flies.
  - Ygg2 2y ago
    I thought it was 2018, it was 2014 instead.
- anonymoushn 2y ago
  "green threads" as discussed in TFA don't require a runtime. The relevant issues are maybe https://github.com/rust-lang/rust/issues/33368 https://github.com/rust-lang/rust/issues/33368 and https://internals.rust-lang.org/t/what-is-the-current-safety-story-for-library-based-stackful-coroutines/14863/16 https://internals.rust-lang.org/t/what-is-the-current-safety...
steveklabnik 2y ago
https://github.com/rust-lang/rfcs/blob/master/text/0230-remove-runtime.md https://github.com/rust-lang/rfcs/blob/master/text/0230-remo... Note this was in 2014, before Rust 1.0.
- hyutmyjahsd 2y ago
  I miss peak rust.
- giancarlostoro 2y ago
  Yes it was a while back, wow 10 years.
jasode 2y ago
>I still remember first hearing about Rust and how it was using Green Threads, [...] I never did look up why they gave up on Green threads It's because green threads have unavoidable performance costs. I collected various threads that has Rust contributors explaining the costs they didn't want to pay: https://news.ycombinator.com/item?id=39062953 https://news.ycombinator.com/item?id=39062953 A language like Go is willing to pay those performance costs because they deliberately sit at a higher level of abstraction than lower-level Rust. Each chose different tradeoffs for different goals: - Golang : green threads worth the perf cost to gain higher productivity (goroutines, channels, etc) for the type of programs that Go programmers like to write - Rust : green threads are not worth the perf cost so as to not sacrifice the best possible performance for the type of programs Rust programmers like to write You can't have a language+runtime that satisfies both camps 100% perfectly. The language designer must choose the tradeoff. But because this tradeoff decision is not blatantly well-known and disseminated ... it also perpetually confuses Language X proponents on why Language Y didn't do the same thing as Language X.
- 01HNNWZ0MV43FF 2y ago
  What do Go channels do that takes advantage of their green threads?
- giancarlostoro 2y ago
  Thank you for this! This is exactly why I posted my comment, I was hoping someone would share some insight about Rust.
zamalek 2y ago
Rust can have green threads. It depends on the reactor. The abstraction sits one level lower than nearly everything else out there.

cube2222 2y ago

This all makes sense but > Green threads are different. The memory of a green thread is allocated on the heap. But all of this comes with a cost: As they aren't managed by the OS, they can't take advantage of multiple cores inherently. But for I/O-bound operations, they are a good fit. this is clearly not true? Am I missing some nuance here, as I'm sure the author knows what they're talking about? Green threads can totally use a multi-threaded runtime, like e.g. Go does, and it works just fine. The main hurdle with them is arguably FFI.

DougBTX 2y ago
The “inherently” means not by default, i.e., the runtime has to support moving green threads between OS threads itself.
- cube2222 2y ago
  Ah, that makes sense, thanks!
- 01HNNWZ0MV43FF 2y ago
  That's not how I use "inherent", maybe they should just say "default" then? But that's like... C is also single threaded by default, what isn't?
  - jayd16 2y ago
    They will never be transparently/fundamentalally managed by the OS alone. The runtime will need to determine how to juggle green threads across multiple OS threads. In that way, this mapping is not inherent. It can be designed around but that itself is a runtime design decision and I would not say it's akin to default vs custom.
  - recursive 2y ago
    With respect, it's not particularly relevant how you use "inherent". It's a standard usage. Rather than asking the whole rest of the world to change, you should probably learn the definition.

bob1029 2y ago

I don't know if this proposal makes a lot of sense. The existing async1/TPL path is stable & predictable. If you find yourself needing more performance, you can reach for hardware Thread instances and use whatever locking/waiting/sharing/context strategies you desire. Anything else is a weird blend of compromises that is going to have caveats that are not immediately obvious for your specific use case. For example, async2 w/ runtime JIT appears to have some tradeoffs with regard to GC & memory usage and the experiment writeup leaves some open-ended questions here[0]. [0]: https://github.com/dotnet/runtimelab/blob/feature/async2-experiment/docs/design/features/runtime-handled-tasks.md#impact-on-gc-heap-size-pauses https://github.com/dotnet/runtimelab/blob/feature/async2-exp...

whaleofatw2022 2y ago
Trying to do clever stuff with threads tends to lead from brow beating, wailing, and gnashing of teeth by various folks who insist to just trust the threadpool.
GordonS 2y ago
> If you find yourself needing more performance, you can reach for hardware Thread instances True for code your team write, but async/await is kind of viral, and many libraries now only have async APIs, which makes them difficult to shoehorn into a threading approach.
- bob1029 2y ago
  > async/await is kind of viral Exactly part of my concern. Hypothetically we've got 2 competing viruses now.
  - GordonS 2y ago
    Surely it's the same "virus"? Unless I'm mistaken, the async2 proposal wouldn't require any code changes, as it essentialy just moves compiler stuff to the runtime.
    - 2y ago

neonsunset 2y ago

For everyone reading this blog post I caution to take the conclusions there with a grain of salt as they are an interpretation of the notes written down here: https://github.com/dotnet/runtimelab/blob/feature/async2-experiment/docs/design/features/runtime-handled-tasks.md https://github.com/dotnet/runtimelab/blob/feature/async2-exp... It is difficult to draw conclusions at the present moment on e.g. memory consumption until the work on this, which is underway, makes it into mainline runtime. It's important to understand that the experiment was first and foremost a research to look into modernizing async implementation, and was a massive success. Now once that is proven, the tuned and polished implementation will be made. Once it is done and makes into a release (it could even be as early as .NET 10), then further review will be possible. With that said, thank you for writing it, .NET tends to be criminally underrated and slept on by the most of the industry, and these help to alleviate it even if just a bit.

hirvi74 2y ago
> .NET tends to be criminally underrated and slept on by the most of the industry I have been programming in .Net/C# for about 8 years now. I absolutely hate Microsoft with every fiber of my being. However, I can't thank them enough for C# (and all the FOSS contributors as well). .Net has been such a pleasant experience that I truly do not want to program in any other language.
- naasking 2y ago
  > I absolutely hate Microsoft with every fiber of my being. Why? I get it back when they were super hostile to Linux and open source, but those days are long past.
  - anonymoushn 2y ago
    Maybe because GP uses Windows sometimes.
  - orphea 2y ago
    They are super hostile to users of their OS, and these days are right now.

ikekkdcjkfke 2y ago

What is the fundemental problem we are trying to solve here? My pet problem with async is that it takes so much syntax, and i wonder why we need to do that. As i understand it it is all about blocking calls taking up OS threads. So i will try to naievly solve it here, and maybe i end up with the same conclusions. When doing a blocking call, the OS thread could just start executing something else transparently and not 'yield' anything to the code, it just stops executing the code and comes back and executes further, no await, no tasks no nothing. Ok, but how did we end up in a thread? The http listener started 4 threads and it's just putting a stack of function pointers and context memory for the threads to eat when they want. There is a separate engine on another OS thread that handles where the threads can start executing ready code again. No Task or await keywords show up in this code. I have no idea how stack traces work but i guess that can just be saved to memory and loaded back again when the threads feel like executing some ready code

irdc 2y ago
The rationale for async/await I keep hearing is that the separate stack required for each thread uses a lot of memory. I’m not sure going full on function colouring is the answer though. A concerted effort for language runtimes to use less stack space and then simply allocating smaller thread stacks sounds to me like a more elegant solution. It certainly is a whole lot easier to debug than a deeply-nested async/await chain.
- iknowstuff 2y ago
  But also avoiding the cost of context switching, and the ability to handle many tasks in embedded environments. https://tweedegolf.nl/en/blog/65/async-rust-vs-rtos-showdown https://tweedegolf.nl/en/blog/65/async-rust-vs-rtos-showdown
  - irdc 2y ago
    This makes it sound like using async/await is more like hand-optimising your code, somewhat similar to rewriting it in assembler. If a way can be found to improve thread switching times (eg. by only switching registers the program actually uses), the fact that the threaded code is easier to debug and reason about becomes a large advantage.

kodablah 2y ago

> And for the transition phase, there has to be interop for async ↔ async2 For those like me who weren't clear whether `async2` was expected to be a real keyword in the final language, it's not[0]. 0 - https://github.com/dotnet/runtime/issues/94620#issuecomment-1883524868 https://github.com/dotnet/runtime/issues/94620#issuecomment-...

augusto-moura 2y ago
async2 looks like such a terrible keyword name, that it didn't even crossed my mind as an option. Only following your linked comment I understood that they used it as a temporary name while testing
TwentyPosts 2y ago
Have you read the article? > As async is already used as an identifier in C#, the team decided to use async2 as a codename for the experiment. If that thing ever makes it into the runtime, it will be called async - so it will be a replacement for the current async implementation.

jayd16 2y ago

I'm encouraged by the talk of improved exception handling and a recognition of how much garbage it creates. It makes it hard to do things in an idiomatic way and keep GC pressure low. I hope they also revisit task state tracking and TaskCanceledException. It feels out of place to use an exception (and unroll the stack) as control flow in the same API they added ValueTask to keep Task handles on the stack.