8 ms·

My rough understanding is that this is similar to async/await in .NET? It’s a shame this article paints a neutral (or even negative) experience with virtual th

My rough understanding is that this is similar to async/await in .NET? It’s a shame this article paints a neutral (or even negative) experience with virtual threads. We rewrote a boring CRUD app that spent 99% of its time waiting the database to respond to be async/await from top-to-bottom. CPU and memory usage went way down on the web server because so many requests could be handled by far fewer threads.

jsiepkes 2y ago
> My rough understanding is that this is similar to async/await in .NET? Well somewhat but also not really. They are green threads like async/await, but it's use is more transparent, unlike async/await. So there are no special "async methods". You just instantiate a "VirtualThread" where you normally instantiate a (kernel) "Thread" and then use it like any other (kernel) thread. This works because for example all blocking IO API will be automatically converted to non-blocking IO underwater.
devjab 2y ago
> My rough understanding is that this is similar to async/await in .NET? Not really. What C# does is sort of similar but it has the disadvantages of splitting your code ecosystem into non-blocking/blocking code. This means you can “accidentally” start your non-blocking code. Something which may cause your relatively simple API to consume a ridiculous amount of resources. It also makes it much more complicated to update and maintain your code as it grows over the years. What is perhaps worse is that C# lacks an interruption model. Java’s approach is much more modern but then it kind of had to be because the JVM already supported structured concurrency from Kotlin. Which means that Java’s “async/await” had to work in a way which wouldn’t break what was already there. Because Java is like that. I think you can sort of view it as another example of how Java has overtaken C# (for now), but I imagine C# will get an improved async/await model in the next couple of years. Neither approach is something you would actually chose if concurrency is important to what you build and you don’t have a legacy reason to continue to build on Java/C# . This is because Go or Erlang would be the obvious choice, but it’s nice that you at least have the option if your organisation is married to a specific language.
- delusional 2y ago
  From what I recall, and this is a while ago so bare with me, Java Virtual Threads still have a lot of pitfalls where the promise of concurrency isn't really fulfilled. I seem to remember that is was some pretty basic operations (like maybe read or something) that caused the thread not to unmount, and therefore just block the underlying os thread. At that point you've just invented the world's most complicated thread pool.
  - za3faran 2y ago
    You're referring to thread pinning, and this is being addressed.
  - mike_hearn 2y ago
    Reading from sockets definitely works. It'd be pretty useless if it didn't. Some operations that don't cause a task switch to another virtual thread are: - If you've called into a native library and back into Java that then blocks. In practice this never happens because Java code doesn't rely on native libraries or frameworks that much and when it does happen it's nearly always in-and-out quickly without callbacks. This can't be fixed by the JVM, however. - File IO. No fundamental problem here, it can be fixed, it's just that not so many programs need tens of thousands of threads doing async file IO. - If you're holding a lock using 'synchronized'. No fundamental problem here, it's just annoying because of how HotSpot is implemented. They're fixing this at the moment. In practice it's mostly the last one that causes issues in real apps. It's not hard to work around, and eventually those workarounds won't be needed anymore.
- szundi 2y ago
  Maybe C# is going to have a new asynv await model but the fragmentation of libs and codes cannot be undone probably. Java has the power that they make relatively more decisions about the language and the libs that they don’t have to fix later. That’s a great value if you’re not building throw-away software but SaaS or something that has to live long.
- za3faran 2y ago
  I would not argue that golang is the obvious choice for concurrency. Java's approach is actually superior to golang's. It takes it a step further by offering structured concurrency[1]. Kotlin's design had no bearing on Java's or the JVM's implementation. C# has an interruption model through CancellationToken as far as I'm aware. [1] https://openjdk.org/jeps/453 https://openjdk.org/jeps/453
xxs 2y ago
>My rough understanding is that this is similar to async/await in .NET? No, the I/O is still blocking with respect to the application code.
kimi 2y ago
It's more like Erlang threads - they appear to be blocking, so existing code will work with zero changes. But you can create a gazillion of them.
he0001 2y ago
> My rough understanding is that this is similar to async/await in .NET? The biggest difference is that C# async/await code is rewritten by the compiler to be able to be async. This means that you see artifacts in the stack that weren’t there when you wrote the code. There are no rewrites with virtual threads and the code is presented on the stack just as you write it. They solve the same problem but in very different ways.
- pansa2 2y ago
  > They solve the same problem but in very different ways. Yes. Async/await is stackless, which leads to the “coloured functions” problem (because it can only suspend function calls one-by-one). Threads are stackful (the whole stack can be suspended at once), which avoids the issue.
- jayd16 2y ago
  There is overlap but they really don't solve the same problem. Cooperative threading has its own advantages and patterns that won't be served by virtual threads.
  - he0001 2y ago
    What patterns does async/await solve which virtual threads don’t?
    - neonsunset 2y ago
      "Green Threads" as implemented in Java is a solution that solves only a single problem - blocking/multiplexing. It does not enable easy concurrency and task/future composition the way C#/JS/Rust do, which offer strictly better and more comprehensive model.
      - za3faran 2y ago
        Structured concurrency[1] offers task composition and more. [1] https://openjdk.org/jeps/453 https://openjdk.org/jeps/453
peteri 2y ago
It's a different model. Microsoft did work on green threads a while ago and decided against continuing. Links: https://github.com/dotnet/runtimelab/issues/2398 https://github.com/dotnet/runtimelab/issues/2398 https://github.com/dotnet/runtimelab/blob/feature/green-threads/docs/design/features/greenthreads.md https://github.com/dotnet/runtimelab/blob/feature/green-thre...
- pjmlp 2y ago
  It should be pointed out, that the main reason they didn't go further was because of added complexity in .NET, when async/await already exists. > Green threads introduce a completely new async programming model. The interaction between green threads and the existing async model is quite complex for .NET developers. For example, invoking async methods from green thread code requires a sync-over-async code pattern that is a very poor choice if the code is executed on a regular thread. Also to note that even the current model is complex enough to warrant a FAQ, https://devblogs.microsoft.com/dotnet/configureawait-faq https://devblogs.microsoft.com/dotnet/configureawait-faq https://github.com/davidfowl/AspNetCoreDiagnosticScenarios/blob/master/AsyncGuidance.md https://github.com/davidfowl/AspNetCoreDiagnosticScenarios/b...
  - neonsunset 2y ago
    This FAQ is a bit outdated in places, and is not something most users should worry about in practice. JVM Green Threads here serve predominantly back-end scenarios, where most of the items on the list are not of concern. This list also exists to address bad habits that carried over from before the tasks were introduced, many years ago. In general, the perceived want of green threads is in part caused by misunderstanding of that one bad article about function coloring. And that one bad article about function coloring also does not talk about the way you do async in C#. Async/await in C# in back-end is a very easy to work with model with explicit understanding where a method returns an operation that promises to complete in the future or not, and composing tasks[0] for easy (massive) concurrency is significantly more idiomatic than doing so with green threads or completable futures that existed in Java before these. And as evidenced by adoption of green threads by large scale Java projects, turns out the failure modes share similarities except green threads end up violating way more expectations and the code author may not have any indication or explicit mechanism to address this, like using AsyncLocal. Also one change to look for is "Runtime Handled Tasks" project in .NET that will replace Roslyn-generated state machine code with runtime-provided suspension mechanism which will only ever suspend at true suspension points where task's execution actually yields asynchronously. So far numbers show at least 5x decrease in overhead, which is massive and will bring performance of computation heavy async paths in line with sync ones: https://github.com/dotnet/runtimelab/blob/feature/async2-experiment/docs/design/features/runtime-handled-tasks.md#benchmark-of-minimum-call-cost-between-async-and-async2-functions https://github.com/dotnet/runtimelab/blob/feature/async2-exp... Note that you were trivially able to have millions of scheduled tasks even before that as they are very lightweight. [0]: e.g. sending requests in parallel is just this using var http = new HttpClient() { BaseAddress = new("https://news.ycombinator.com/news") }; var requests = Enumerable .Range(1, 4) .Select(n => $"?p={n}") .Select(http.GetStringAsync); var pages = await Task.WhenAll(requests);
fulafel 2y ago
Can you expand on how the benefit in your rewrite came about? Threads don't consume CPU when they're waiting for the DB, after all. And threads share memory with each other. (I guess scaling to ridiculous levels you could be approaching trouble if you have O(100k) outstanding DB queries per application server, hope you have a DB that can handle millions of oustanding DB queries then!)
- segfaltnh 2y ago
  In large numbers the cost of switching between threads does consume CPU while they're waiting for the database. This is why green threads exist, to have large numbers of in flight work executing over a smaller number of OS threads.
  - fulafel 2y ago
    When using OS threads, there's no switching when they are waiting for a socket (db connection). The OS knows to wake the thread up only when there's something new to see on the connection.
    - deleted 2y ago
      [deleted]
    - kbolino 2y ago
      Both sides of a sleep/awake transition with conventional blocking system calls involve heavyweight context switches: the CPU protection level changes and the thread registers get saved out or loaded back in.
      - fulafel 2y ago
        Yes, but these don't happen while waiting.
        deleted 2y ago
        [deleted]

jsiepkes 2y ago

> My rough understanding is that this is similar to async/await in .NET? Well somewhat but also not really. They are green threads like async/await, but it's use is more transparent, unlike async/await. So there are no special "async methods". You just instantiate a "VirtualThread" where you normally instantiate a (kernel) "Thread" and then use it like any other (kernel) thread. This works because for example all blocking IO API will be automatically converted to non-blocking IO underwater.

devjab 2y ago

> My rough understanding is that this is similar to async/await in .NET? Not really. What C# does is sort of similar but it has the disadvantages of splitting your code ecosystem into non-blocking/blocking code. This means you can “accidentally” start your non-blocking code. Something which may cause your relatively simple API to consume a ridiculous amount of resources. It also makes it much more complicated to update and maintain your code as it grows over the years. What is perhaps worse is that C# lacks an interruption model. Java’s approach is much more modern but then it kind of had to be because the JVM already supported structured concurrency from Kotlin. Which means that Java’s “async/await” had to work in a way which wouldn’t break what was already there. Because Java is like that. I think you can sort of view it as another example of how Java has overtaken C# (for now), but I imagine C# will get an improved async/await model in the next couple of years. Neither approach is something you would actually chose if concurrency is important to what you build and you don’t have a legacy reason to continue to build on Java/C# . This is because Go or Erlang would be the obvious choice, but it’s nice that you at least have the option if your organisation is married to a specific language.

delusional 2y ago
From what I recall, and this is a while ago so bare with me, Java Virtual Threads still have a lot of pitfalls where the promise of concurrency isn't really fulfilled. I seem to remember that is was some pretty basic operations (like maybe read or something) that caused the thread not to unmount, and therefore just block the underlying os thread. At that point you've just invented the world's most complicated thread pool.
- za3faran 2y ago
  You're referring to thread pinning, and this is being addressed.
- mike_hearn 2y ago
  Reading from sockets definitely works. It'd be pretty useless if it didn't. Some operations that don't cause a task switch to another virtual thread are: - If you've called into a native library and back into Java that then blocks. In practice this never happens because Java code doesn't rely on native libraries or frameworks that much and when it does happen it's nearly always in-and-out quickly without callbacks. This can't be fixed by the JVM, however. - File IO. No fundamental problem here, it can be fixed, it's just that not so many programs need tens of thousands of threads doing async file IO. - If you're holding a lock using 'synchronized'. No fundamental problem here, it's just annoying because of how HotSpot is implemented. They're fixing this at the moment. In practice it's mostly the last one that causes issues in real apps. It's not hard to work around, and eventually those workarounds won't be needed anymore.
szundi 2y ago
Maybe C# is going to have a new asynv await model but the fragmentation of libs and codes cannot be undone probably. Java has the power that they make relatively more decisions about the language and the libs that they don’t have to fix later. That’s a great value if you’re not building throw-away software but SaaS or something that has to live long.
za3faran 2y ago
I would not argue that golang is the obvious choice for concurrency. Java's approach is actually superior to golang's. It takes it a step further by offering structured concurrency[1]. Kotlin's design had no bearing on Java's or the JVM's implementation. C# has an interruption model through CancellationToken as far as I'm aware. [1] https://openjdk.org/jeps/453 https://openjdk.org/jeps/453

xxs 2y ago

>My rough understanding is that this is similar to async/await in .NET? No, the I/O is still blocking with respect to the application code.

kimi 2y ago

It's more like Erlang threads - they appear to be blocking, so existing code will work with zero changes. But you can create a gazillion of them.

he0001 2y ago

> My rough understanding is that this is similar to async/await in .NET? The biggest difference is that C# async/await code is rewritten by the compiler to be able to be async. This means that you see artifacts in the stack that weren’t there when you wrote the code. There are no rewrites with virtual threads and the code is presented on the stack just as you write it. They solve the same problem but in very different ways.

pansa2 2y ago
> They solve the same problem but in very different ways. Yes. Async/await is stackless, which leads to the “coloured functions” problem (because it can only suspend function calls one-by-one). Threads are stackful (the whole stack can be suspended at once), which avoids the issue.
jayd16 2y ago
There is overlap but they really don't solve the same problem. Cooperative threading has its own advantages and patterns that won't be served by virtual threads.
- he0001 2y ago
  What patterns does async/await solve which virtual threads don’t?
  - neonsunset 2y ago
    "Green Threads" as implemented in Java is a solution that solves only a single problem - blocking/multiplexing. It does not enable easy concurrency and task/future composition the way C#/JS/Rust do, which offer strictly better and more comprehensive model.
    - za3faran 2y ago
      Structured concurrency[1] offers task composition and more. [1] https://openjdk.org/jeps/453 https://openjdk.org/jeps/453

peteri 2y ago

It's a different model. Microsoft did work on green threads a while ago and decided against continuing. Links: https://github.com/dotnet/runtimelab/issues/2398 https://github.com/dotnet/runtimelab/issues/2398 https://github.com/dotnet/runtimelab/blob/feature/green-threads/docs/design/features/greenthreads.md https://github.com/dotnet/runtimelab/blob/feature/green-thre...

pjmlp 2y ago
It should be pointed out, that the main reason they didn't go further was because of added complexity in .NET, when async/await already exists. > Green threads introduce a completely new async programming model. The interaction between green threads and the existing async model is quite complex for .NET developers. For example, invoking async methods from green thread code requires a sync-over-async code pattern that is a very poor choice if the code is executed on a regular thread. Also to note that even the current model is complex enough to warrant a FAQ, https://devblogs.microsoft.com/dotnet/configureawait-faq https://devblogs.microsoft.com/dotnet/configureawait-faq https://github.com/davidfowl/AspNetCoreDiagnosticScenarios/blob/master/AsyncGuidance.md https://github.com/davidfowl/AspNetCoreDiagnosticScenarios/b...
- neonsunset 2y ago
  This FAQ is a bit outdated in places, and is not something most users should worry about in practice. JVM Green Threads here serve predominantly back-end scenarios, where most of the items on the list are not of concern. This list also exists to address bad habits that carried over from before the tasks were introduced, many years ago. In general, the perceived want of green threads is in part caused by misunderstanding of that one bad article about function coloring. And that one bad article about function coloring also does not talk about the way you do async in C#. Async/await in C# in back-end is a very easy to work with model with explicit understanding where a method returns an operation that promises to complete in the future or not, and composing tasks[0] for easy (massive) concurrency is significantly more idiomatic than doing so with green threads or completable futures that existed in Java before these. And as evidenced by adoption of green threads by large scale Java projects, turns out the failure modes share similarities except green threads end up violating way more expectations and the code author may not have any indication or explicit mechanism to address this, like using AsyncLocal. Also one change to look for is "Runtime Handled Tasks" project in .NET that will replace Roslyn-generated state machine code with runtime-provided suspension mechanism which will only ever suspend at true suspension points where task's execution actually yields asynchronously. So far numbers show at least 5x decrease in overhead, which is massive and will bring performance of computation heavy async paths in line with sync ones: https://github.com/dotnet/runtimelab/blob/feature/async2-experiment/docs/design/features/runtime-handled-tasks.md#benchmark-of-minimum-call-cost-between-async-and-async2-functions https://github.com/dotnet/runtimelab/blob/feature/async2-exp... Note that you were trivially able to have millions of scheduled tasks even before that as they are very lightweight. [0]: e.g. sending requests in parallel is just this using var http = new HttpClient() { BaseAddress = new("https://news.ycombinator.com/news") }; var requests = Enumerable .Range(1, 4) .Select(n => $"?p={n}") .Select(http.GetStringAsync); var pages = await Task.WhenAll(requests);

fulafel 2y ago

Can you expand on how the benefit in your rewrite came about? Threads don't consume CPU when they're waiting for the DB, after all. And threads share memory with each other. (I guess scaling to ridiculous levels you could be approaching trouble if you have O(100k) outstanding DB queries per application server, hope you have a DB that can handle millions of oustanding DB queries then!)

segfaltnh 2y ago
In large numbers the cost of switching between threads does consume CPU while they're waiting for the database. This is why green threads exist, to have large numbers of in flight work executing over a smaller number of OS threads.
- fulafel 2y ago
  When using OS threads, there's no switching when they are waiting for a socket (db connection). The OS knows to wake the thread up only when there's something new to see on the connection.
  - deleted 2y ago
    [deleted]
  - kbolino 2y ago
    Both sides of a sleep/awake transition with conventional blocking system calls involve heavyweight context switches: the CPU protection level changes and the thread registers get saved out or loaded back in.
    - fulafel 2y ago
      Yes, but these don't happen while waiting.
      - deleted 2y ago
        [deleted]