8 ms·

What is the virtual thread / event loop pattern seeking to optimize? Is it context switching? A number of years ago I remember trying to have a sane discussion

What is the virtual thread / event loop pattern seeking to optimize? Is it context switching? A number of years ago I remember trying to have a sane discussion about “non blocking” and I remember saying “something” will block eventually no matter what… anything from the buffer being full on the NIC to your cpu being at anything less than 100%. Does it shake out to any real advantage?

kevingadd 2y ago
One of the main reasons to do virtual threads is that it allows you to write naive "thread per request" code and still scale up significantly without hitting the kind of scaling limits you would with OS threads.
- hashmash 2y ago
  The problem with the naïve design is that even with virtual threads, you risk running out of (heap) memory if the threads ever block. Each task makes a bit of progress, allocates some objects, and then lets another one do the same thing. With virtual threads, you can limit the damage by using a semaphore, but you still need to tune the size. This isn't much different than sizing a traditional thread pool, and so I'm not sure what benefit virtual threads will really have in practice. You're swapping one config for another.
  - immibis 2y ago
    Async does exactly the same by the way.
  - initplus 2y ago
    The benefits from virtual threads come from the simple API that it presents to the programmer. It's not a performance optimization.
    - hashmash 2y ago
      But that same benefit was always available with platform threads -- a simple API. What is the real gain by using virtual threads? It's either going to be performance or memory utilization.
      - groestl 2y ago
        It's combining the benefits from async models (state machines separated from os threads, thus more optimal for I/O bound workload), with the benefits from proper threading models (namely the simpler human interface). Memory utilization & performance is going to be similar to the async callback mess.
        hashmash 2y ago
        Why is an async model better than using OS threads for an I/O bound workload? The OS is doing async stuff internally and shielding the complexity with threads. With virtual threads this work has shifted to the JVM. Can the JVM do threads better than the OS?
        adgjlsfhk1 2y ago
        it can do a much better job because there isn't a security boundary. OS thread scheduling requires sys calls and invalidate a bunch of cache to prevent timing leaks
        zokier 2y ago
        > Can the JVM do threads better than the OS? Yes. The JVM has far more opportunities for optimizing threads because it doesn't need to uphold 50 years of accumulated invariants and compatibility that current OSes do, and JVM has more visibilty on the application internals.
        mrsilencedogood 2y ago
        "Why is an async model better than using OS threads for an I/O bound workload?" Because evented/callback-driven code is a nightmare to reason about and breaks lots of very basic tools, like the humble stack trace. Another big thing for me is resource management - try/finally don't work across callback boundaries, but do work within a virtual thread. I recently ported a netty-based evented system to virtual threads and a very long-standing issue - resource leakage - turned into one very nice try/finally block.
  - packetlost 2y ago
    Yeah, and it's generally good to be RAM limited instead of CPU, no? The alternative is blowing a bunch of time on syscalls and OS scheduler overhead. Also the virtual threads run on a "traditional" thread pool to my understanding, so you can just tweak the number of worker threads to cap the total concurrency. The benefit is it's overall more efficient (in the general case) and lets you write linear blocking code (as opposed to function coloring). You don't have to use it, but it's nice that it's there. Now hopefully Valhalla actually makes it in eventually
    - hashmash 2y ago
      The OS scheduler is still there (for the carrier threads), but now you've added on top of that FJ pool based scheduler overhead. Although virtual threads don't have the syscall overhead when they block, there's a new cost caused by allocating the internal continuation object, and copying state into it. This puts more pressure on the garbage collector. Context switching cost due to CPU cache thrashing doesn't go away regardless of which type of thread you're using. I've not yet seen a study that shows that virtual threads offer a huge benefit. The Open Liberty study suggests that they're worse than the existing platform threads.
      - zokier 2y ago
        > The OS scheduler is still there (for the carrier threads), but now you've added on top of that FJ pool based scheduler overhead. Ideally carrier threads would be pinned to isolated cpu cores, which removes most aspects of OS scheduler from the picture
      - zokier 2y ago
        > I've not yet seen a study that shows that virtual threads offer a huge benefit. Not exactly Java virtual threads, but a study on how userland threads beat kernel threads. https://cs.uwaterloo.ca/~mkarsten/papers/sigmetrics2020.html https://cs.uwaterloo.ca/~mkarsten/papers/sigmetrics2020.html For quick results, check figures 11 and 15 from the (preprint) paper. Userland threads ("fred") have ~50% higher throughput while having orders of magnitude better latency at high load levels, in a real-world application (memcached).
  - dikei 2y ago
    > The problem with the naïve design is that even with virtual threads, you risk running out of (heap) memory if the threads ever block. The key with virtual threads is they are so light weight that you can have thousands of them running concurrently: even when they block for I/O, it doesn't matter. It's similar to light weight coroutine in other language like Go or Kotlin.
  - imtringued 2y ago
    What you are complaining about has nothing to do with thread pools or virtual threads. You're pointing out the fact that more parallelism will also need more hardware and that a finite hardware budget will need a back pressure strategy to keep resource consumption within a limit. While you might be correct that "sizing a traditional thread pool" is a back pressure strategy that can be applied to virtual threads, the problem with it is that IO bound threads will prevent CPU bound threads from making progress. You don't want to apply back pressure based on the number of tasks. You want back pressure to be in response to resource utilization, so that enough tasks get scheduled to max out the hardware. This is a common problem with people using Java parallel streams, because they by default share a single global thread pool and the way to use your own thread pool is also extremely counterintuitive, because it essentially relies on some implicit thread local magic to choose to distribute the stream in the thread pool that the parallel stream was launched on, instead of passing it as a parameter. It would be best if people came up with more dynamic back pressure strategies, because this is a more general problem that goes way beyond thread pools. In fact, one of the key problems of automatic parallelization is deciding at what point there is too much parallelization.
fzeindl 2y ago
Does it shake out to any real advantage? To put it shortly: Writing single-threaded blocking code is far easier for most people and has many other benefits, like more understandable and readable programs: https://www.youtube.com/watch?v=449j7oKQVkc https://www.youtube.com/watch?v=449j7oKQVkc The main reason why non-blocking IO with it's style of intertwining concurrency and algorithms came along is that starting a thread for every request was too expensive. With virtual threads that problem is eliminated so we can go back to writing blocking code.
- nlitened 2y ago
  > is far easier for most people I’d say that writing single-threaded code is far easier for _all_ people, even async code experts :) Also, single-threaded code is supported by programming language facilities: you have a proper call stack, thread-local vars, exceptions bubbling up, structured concurrency, simple resource management (RAII, try-with-resources, defer). Easy to reason and debug on language level. Async runtimes are always complicated, filled with leaky abstractions, it’s like another language that one has to learn in addition, but with a less thought-out, ad-hoc design. Difficult to reason and debug, especially in edge cases
  - bheadmaster 2y ago
    > Async runtimes are always complicated, filled with leaky abstractions, it’s like another language that one has to learn in addition, but with a less thought-out, ad-hoc design. Difficult to reason and debug, especially in edge cases Async runtimes themselves are simply attempts to bolt-on green threads on top of a language that doesn't support them on a language level. In JavaScript, async/await uses Promises to enable callback-code to interact with key language features like try/catch, for/while/break, return, etc. In Python, async/await is just syntax sugar for coroutines, which are again just syntax sugar for CPS-style classes with methods split at each "yield". Not sure about Rust, but it probably also uses some Rust macro magic to do something similar.
    - logicchains 2y ago
      >Async runtimes themselves are simply attempts to bolt-on green threads on top of a language that doesn't support them on a language level. Haskell supports async code while also supporting green threads on a language level, and the async code has most of the same issues as async code in any other languages.
gregopet 2y ago
It's a brave attempt to release the programmer from worrying or even thinking about thread pools and blocking code. Java has gone all in - they even cancelled a non-blocking rewrite of their database driver architecture because why have that if you won't have to worry about blocking code? And the JVM really is a marvel of engineering, it's really really good at what it does, so what team to better pull this off? So far, they're not quite there yet: the issue of "thread pinning" is something developers still have to be aware of. I hear the newest JVM version has removed a few more cases where it happens, but will we ever truly 100% not have to care about all that anymore? I have to say things are already pretty awesome however. If you avoid the few thread pinning causes (and can avoid libraries that use them - although most of not all modern libraries have already adapted), you can write really clean code. We had to rewrite an old app that made a huge mess tracking a process where multiple event sources can act independently, and virtual threads seemed the perfect thing for it. Now our business logic looks more like a game loop and not the complicated mix of pollers, request handlers, intermediate state persisters (with their endless thirst for various mappers) and whatnot that it was before (granted, all those things weren't there just because of threading.. the previous version was really really shitily written). It's true that virtual threads sometimes hurt performance (since their main benefit is cleaner simpler code). Not by much, usually, but a precisely written and carefully tuned piece of performance critical code can often still do things better than automatic threading code. And as a fun aside, some very popular libraries assumed the developer is using thread pools (before virtual threads, which non trivial Java app didn't? - ok nobody answer that, I'm sure there are cases :D) so these libraries had performance tricks (ab)using thread pool code specifics. So that's another possible performance issue with virtual threads - like always with performance of course: don't just assume, try it and measure! :P
- immibis 2y ago
  So... What is it seeking to optimize? Why did you need a thread pool before but not any more? What resource was exhausted to prevent you from putting every request on a thread?
  - davidgay 2y ago
    A thread per request has a high risk of overcommitting on CPU use, leading to a different set of problems. Virtual threads are scheduled on a fixed-size (based on number of cores) underlying (non-virtual) thread pool to avoid this problem.
lmm 2y ago
> I remember saying “something” will block eventually no matter what… anything from the buffer being full on the NIC to your cpu being at anything less than 100%. Nope. You can go async all the way down, right to the electrical signals if you want. We usually impose some amount of synchronous clocking/polling for sanity, at various levels, but you don't have to; the world is not synchronised, the fastest way to respond to a stimulus will always be to respond when it happens. > Does it shake out to any real advantage? Of course it does - did you miss the whole C10K discussions 20+ years ago? Whether it matters for your business is another question, but you can absolutely get a lot more throughput by being nonblocking, and if you're doing request-response across the Internet you generally can't afford not to.
duped 2y ago
imo the biggest difference between "virtual" threads in a managed runtime and "os" threads is that the latter uses a fixed size stack whereas the former is allowed to resize, it can grow on demand and shrink under pressure. When you spawn an OS thread you are paying at worst the full cost of it, and at best the max depth seen so far in the program, and stack overflows can happen even if the program is written correctly. Whereas a virtual thread can grow the stack to be exactly the size it needs at any point, and when GC runs it can rewrite pointers to any data on the stack safely. Virtual/green/user space threads aka stackful coroutines have proven to be an excellent tool for scaling concurrency in real programs, while threads and processes have always played catchup. > “something” will block eventually no matter what… The point is to allow everything else to make progress while that resource is busy. --- At a broader scale, as a programming model it lets you architect programs that are designed to scale horizontally. With the commodization of compute in the cloud that means it's very easy to write a program that can be distributed as i/o demand increases. In principle, a "virtual" thread could be spawned on a different machine entirely.
chipdart 2y ago
> What is the virtual thread / event loop pattern seeking to optimize? Is it context switching? Throughput. Some workloads are not CPU-bound or memory-bound, and spend the bulk of their time waiting for external processes to make data available. If your workloads are expected to stay idle while waiting for external events, you can switch to other tasks while you wait for those external events to trigger. This is particularly convenient if the other tasks you're hoping to run are also tasks that are bound to stay idle while waiting for external events. One of the textbook scenarios that suits this pattern well is making HTTP requests. Another one is request handlers, such as the controller pattern used so often in HTTP servers. Perhaps the poster child of this pattern is Node.js. It might not be the performance king and might be single-threaded, but it features in the top spots in performance benchmarks such as TechEmpower's. Node.js is also highly favoured in function-as-a-service applications, as it's event-driven architecture is well suited for applications involving a hefty dose of network calls running on memory- and CPU-constrained systems.
pron 2y ago
No, it optimises hardware utilisation by simply allowing more tasks to concurrently make progress. This allows throughput to reach the maximum the hardware allows. See https://youtu.be/07V08SB1l8c https://youtu.be/07V08SB1l8c.
frevib 2y ago
They indeed optimize thread context switching. Taking the thread on and off the CPU is becoming expensive when there are thousands of threads. You are right that everything blocks, even when going to L1 cache you have to wait 1 nanoseconds. But blocking in this context means waiting for “real” IO like a network request or spinning disk access. Virtual threads take away the problem that the thread sits there doing nothing for a while as it is waiting for data, before it is context switched. Virtual threads won’t improve CPU-bound blocking. There the thread is actually occupying the CPU, so there is no problem of the thread doing nothing as with IO-bound blocking.
kbolino 2y ago
The hardware now is just as concurrent/parallel as the software. High-end NVMe SSDs and server-grade NICs can do hundreds to thousands of things simultaneously. Even if one lane does get blocked, there are other lanes which are open.

kevingadd 2y ago

One of the main reasons to do virtual threads is that it allows you to write naive "thread per request" code and still scale up significantly without hitting the kind of scaling limits you would with OS threads.

hashmash 2y ago
The problem with the naïve design is that even with virtual threads, you risk running out of (heap) memory if the threads ever block. Each task makes a bit of progress, allocates some objects, and then lets another one do the same thing. With virtual threads, you can limit the damage by using a semaphore, but you still need to tune the size. This isn't much different than sizing a traditional thread pool, and so I'm not sure what benefit virtual threads will really have in practice. You're swapping one config for another.
- immibis 2y ago
  Async does exactly the same by the way.
- initplus 2y ago
  The benefits from virtual threads come from the simple API that it presents to the programmer. It's not a performance optimization.
  - hashmash 2y ago
    But that same benefit was always available with platform threads -- a simple API. What is the real gain by using virtual threads? It's either going to be performance or memory utilization.
    - groestl 2y ago
      It's combining the benefits from async models (state machines separated from os threads, thus more optimal for I/O bound workload), with the benefits from proper threading models (namely the simpler human interface). Memory utilization & performance is going to be similar to the async callback mess.
      - hashmash 2y ago
        Why is an async model better than using OS threads for an I/O bound workload? The OS is doing async stuff internally and shielding the complexity with threads. With virtual threads this work has shifted to the JVM. Can the JVM do threads better than the OS?
        adgjlsfhk1 2y ago
        it can do a much better job because there isn't a security boundary. OS thread scheduling requires sys calls and invalidate a bunch of cache to prevent timing leaks
        zokier 2y ago
        > Can the JVM do threads better than the OS? Yes. The JVM has far more opportunities for optimizing threads because it doesn't need to uphold 50 years of accumulated invariants and compatibility that current OSes do, and JVM has more visibilty on the application internals.
        mrsilencedogood 2y ago
        "Why is an async model better than using OS threads for an I/O bound workload?" Because evented/callback-driven code is a nightmare to reason about and breaks lots of very basic tools, like the humble stack trace. Another big thing for me is resource management - try/finally don't work across callback boundaries, but do work within a virtual thread. I recently ported a netty-based evented system to virtual threads and a very long-standing issue - resource leakage - turned into one very nice try/finally block.
- packetlost 2y ago
  Yeah, and it's generally good to be RAM limited instead of CPU, no? The alternative is blowing a bunch of time on syscalls and OS scheduler overhead. Also the virtual threads run on a "traditional" thread pool to my understanding, so you can just tweak the number of worker threads to cap the total concurrency. The benefit is it's overall more efficient (in the general case) and lets you write linear blocking code (as opposed to function coloring). You don't have to use it, but it's nice that it's there. Now hopefully Valhalla actually makes it in eventually
  - hashmash 2y ago
    The OS scheduler is still there (for the carrier threads), but now you've added on top of that FJ pool based scheduler overhead. Although virtual threads don't have the syscall overhead when they block, there's a new cost caused by allocating the internal continuation object, and copying state into it. This puts more pressure on the garbage collector. Context switching cost due to CPU cache thrashing doesn't go away regardless of which type of thread you're using. I've not yet seen a study that shows that virtual threads offer a huge benefit. The Open Liberty study suggests that they're worse than the existing platform threads.
    - zokier 2y ago
      > The OS scheduler is still there (for the carrier threads), but now you've added on top of that FJ pool based scheduler overhead. Ideally carrier threads would be pinned to isolated cpu cores, which removes most aspects of OS scheduler from the picture
    - zokier 2y ago
      > I've not yet seen a study that shows that virtual threads offer a huge benefit. Not exactly Java virtual threads, but a study on how userland threads beat kernel threads. https://cs.uwaterloo.ca/~mkarsten/papers/sigmetrics2020.html https://cs.uwaterloo.ca/~mkarsten/papers/sigmetrics2020.html For quick results, check figures 11 and 15 from the (preprint) paper. Userland threads ("fred") have ~50% higher throughput while having orders of magnitude better latency at high load levels, in a real-world application (memcached).
- dikei 2y ago
  > The problem with the naïve design is that even with virtual threads, you risk running out of (heap) memory if the threads ever block. The key with virtual threads is they are so light weight that you can have thousands of them running concurrently: even when they block for I/O, it doesn't matter. It's similar to light weight coroutine in other language like Go or Kotlin.
- imtringued 2y ago
  What you are complaining about has nothing to do with thread pools or virtual threads. You're pointing out the fact that more parallelism will also need more hardware and that a finite hardware budget will need a back pressure strategy to keep resource consumption within a limit. While you might be correct that "sizing a traditional thread pool" is a back pressure strategy that can be applied to virtual threads, the problem with it is that IO bound threads will prevent CPU bound threads from making progress. You don't want to apply back pressure based on the number of tasks. You want back pressure to be in response to resource utilization, so that enough tasks get scheduled to max out the hardware. This is a common problem with people using Java parallel streams, because they by default share a single global thread pool and the way to use your own thread pool is also extremely counterintuitive, because it essentially relies on some implicit thread local magic to choose to distribute the stream in the thread pool that the parallel stream was launched on, instead of passing it as a parameter. It would be best if people came up with more dynamic back pressure strategies, because this is a more general problem that goes way beyond thread pools. In fact, one of the key problems of automatic parallelization is deciding at what point there is too much parallelization.

fzeindl 2y ago

Does it shake out to any real advantage? To put it shortly: Writing single-threaded blocking code is far easier for most people and has many other benefits, like more understandable and readable programs: https://www.youtube.com/watch?v=449j7oKQVkc https://www.youtube.com/watch?v=449j7oKQVkc The main reason why non-blocking IO with it's style of intertwining concurrency and algorithms came along is that starting a thread for every request was too expensive. With virtual threads that problem is eliminated so we can go back to writing blocking code.

nlitened 2y ago
> is far easier for most people I’d say that writing single-threaded code is far easier for _all_ people, even async code experts :) Also, single-threaded code is supported by programming language facilities: you have a proper call stack, thread-local vars, exceptions bubbling up, structured concurrency, simple resource management (RAII, try-with-resources, defer). Easy to reason and debug on language level. Async runtimes are always complicated, filled with leaky abstractions, it’s like another language that one has to learn in addition, but with a less thought-out, ad-hoc design. Difficult to reason and debug, especially in edge cases
- bheadmaster 2y ago
  > Async runtimes are always complicated, filled with leaky abstractions, it’s like another language that one has to learn in addition, but with a less thought-out, ad-hoc design. Difficult to reason and debug, especially in edge cases Async runtimes themselves are simply attempts to bolt-on green threads on top of a language that doesn't support them on a language level. In JavaScript, async/await uses Promises to enable callback-code to interact with key language features like try/catch, for/while/break, return, etc. In Python, async/await is just syntax sugar for coroutines, which are again just syntax sugar for CPS-style classes with methods split at each "yield". Not sure about Rust, but it probably also uses some Rust macro magic to do something similar.
  - logicchains 2y ago
    >Async runtimes themselves are simply attempts to bolt-on green threads on top of a language that doesn't support them on a language level. Haskell supports async code while also supporting green threads on a language level, and the async code has most of the same issues as async code in any other languages.

gregopet 2y ago

It's a brave attempt to release the programmer from worrying or even thinking about thread pools and blocking code. Java has gone all in - they even cancelled a non-blocking rewrite of their database driver architecture because why have that if you won't have to worry about blocking code? And the JVM really is a marvel of engineering, it's really really good at what it does, so what team to better pull this off? So far, they're not quite there yet: the issue of "thread pinning" is something developers still have to be aware of. I hear the newest JVM version has removed a few more cases where it happens, but will we ever truly 100% not have to care about all that anymore? I have to say things are already pretty awesome however. If you avoid the few thread pinning causes (and can avoid libraries that use them - although most of not all modern libraries have already adapted), you can write really clean code. We had to rewrite an old app that made a huge mess tracking a process where multiple event sources can act independently, and virtual threads seemed the perfect thing for it. Now our business logic looks more like a game loop and not the complicated mix of pollers, request handlers, intermediate state persisters (with their endless thirst for various mappers) and whatnot that it was before (granted, all those things weren't there just because of threading.. the previous version was really really shitily written). It's true that virtual threads sometimes hurt performance (since their main benefit is cleaner simpler code). Not by much, usually, but a precisely written and carefully tuned piece of performance critical code can often still do things better than automatic threading code. And as a fun aside, some very popular libraries assumed the developer is using thread pools (before virtual threads, which non trivial Java app didn't? - ok nobody answer that, I'm sure there are cases :D) so these libraries had performance tricks (ab)using thread pool code specifics. So that's another possible performance issue with virtual threads - like always with performance of course: don't just assume, try it and measure! :P

immibis 2y ago
So... What is it seeking to optimize? Why did you need a thread pool before but not any more? What resource was exhausted to prevent you from putting every request on a thread?
- davidgay 2y ago
  A thread per request has a high risk of overcommitting on CPU use, leading to a different set of problems. Virtual threads are scheduled on a fixed-size (based on number of cores) underlying (non-virtual) thread pool to avoid this problem.

lmm 2y ago

> I remember saying “something” will block eventually no matter what… anything from the buffer being full on the NIC to your cpu being at anything less than 100%. Nope. You can go async all the way down, right to the electrical signals if you want. We usually impose some amount of synchronous clocking/polling for sanity, at various levels, but you don't have to; the world is not synchronised, the fastest way to respond to a stimulus will always be to respond when it happens. > Does it shake out to any real advantage? Of course it does - did you miss the whole C10K discussions 20+ years ago? Whether it matters for your business is another question, but you can absolutely get a lot more throughput by being nonblocking, and if you're doing request-response across the Internet you generally can't afford not to.

duped 2y ago

imo the biggest difference between "virtual" threads in a managed runtime and "os" threads is that the latter uses a fixed size stack whereas the former is allowed to resize, it can grow on demand and shrink under pressure. When you spawn an OS thread you are paying at worst the full cost of it, and at best the max depth seen so far in the program, and stack overflows can happen even if the program is written correctly. Whereas a virtual thread can grow the stack to be exactly the size it needs at any point, and when GC runs it can rewrite pointers to any data on the stack safely. Virtual/green/user space threads aka stackful coroutines have proven to be an excellent tool for scaling concurrency in real programs, while threads and processes have always played catchup. > “something” will block eventually no matter what… The point is to allow everything else to make progress while that resource is busy. --- At a broader scale, as a programming model it lets you architect programs that are designed to scale horizontally. With the commodization of compute in the cloud that means it's very easy to write a program that can be distributed as i/o demand increases. In principle, a "virtual" thread could be spawned on a different machine entirely.

chipdart 2y ago

> What is the virtual thread / event loop pattern seeking to optimize? Is it context switching? Throughput. Some workloads are not CPU-bound or memory-bound, and spend the bulk of their time waiting for external processes to make data available. If your workloads are expected to stay idle while waiting for external events, you can switch to other tasks while you wait for those external events to trigger. This is particularly convenient if the other tasks you're hoping to run are also tasks that are bound to stay idle while waiting for external events. One of the textbook scenarios that suits this pattern well is making HTTP requests. Another one is request handlers, such as the controller pattern used so often in HTTP servers. Perhaps the poster child of this pattern is Node.js. It might not be the performance king and might be single-threaded, but it features in the top spots in performance benchmarks such as TechEmpower's. Node.js is also highly favoured in function-as-a-service applications, as it's event-driven architecture is well suited for applications involving a hefty dose of network calls running on memory- and CPU-constrained systems.

pron 2y ago

No, it optimises hardware utilisation by simply allowing more tasks to concurrently make progress. This allows throughput to reach the maximum the hardware allows. See https://youtu.be/07V08SB1l8c https://youtu.be/07V08SB1l8c.

frevib 2y ago

They indeed optimize thread context switching. Taking the thread on and off the CPU is becoming expensive when there are thousands of threads. You are right that everything blocks, even when going to L1 cache you have to wait 1 nanoseconds. But blocking in this context means waiting for “real” IO like a network request or spinning disk access. Virtual threads take away the problem that the thread sits there doing nothing for a while as it is waiting for data, before it is context switched. Virtual threads won’t improve CPU-bound blocking. There the thread is actually occupying the CPU, so there is no problem of the thread doing nothing as with IO-bound blocking.

kbolino 2y ago

The hardware now is just as concurrent/parallel as the software. High-end NVMe SSDs and server-grade NICs can do hundreds to thousands of things simultaneously. Even if one lane does get blocked, there are other lanes which are open.