7 ms·

Excellent! With virtual threads, all the blocking code I wrote is efficient now :) Less-joking: I'm so excited for this to start getting adoption. For a pre-re

Excellent! With virtual threads, all the blocking code I wrote is efficient now :) Less-joking: I'm so excited for this to start getting adoption. For a pre-release/preview feature, Loom already has SO much traction, with most major frameworks already having support for running app/user code on virtual threads and one notable project (Helidon Nima) replacing netty with their own solution based on blocking IO and virtual threads. Now I want to see the community run with it. I've always thought async IO was just plain gross. Python's implementation is so yucky that after using it for one project I decided that I'd rather DIY it with multiprocessing than use async again. (I don't have any more constructive feedback than that, my apologies, it was a while ago so I don't remember the specifics but what has lasted is the sour taste of the sum of all the problems we had with it - perhaps notably that only about 2 people on my dev team of 5 actually understood the async paradigm). netty did it fine. I've built multiple projects on top of netty and it's fine. I like event-based-async more than async-await-based-async. But it's still a headache and notably I really rather missed the kinds of guarantees you can get (in blocking code) by wrapping the block in try-catch-finally (to e.g. guarantee resources get freed or that two counters, say a requests-in and a requests-out, are guaranteed to match up). But dang am I excited to not do that anymore. I have one specific project that I'm planning to port from async to blocking+virtualthreads that I expect to greatly simplify the code. It has a lot of requests it makes back and forth (it has to manually resolve DNS queries among other things) so there's good chunks of 50-200 ms where I have to either yield (and has gross async code that yields and resumes all the heck over the place) or block the thread for human-noticeable chunks of time (also very gross of course!).

eastbound 3y ago
And it’s LTS! When is the book and the certification exam coming out, does anyone know?
smallerfish 3y ago
Can you explain why you're excited about with virtual threads? I get that they improve throughput in extremely high pressure apps, but the JVM's current threading isn't exactly a slouch. Java's used in HFT shops, and more generally in fintech where performance matters.
- mr_tristan 3y ago
  I can’t speak for the OP, but this makes it much easier to write code that uses threads that wait on IO, and just let the underlying system (VM + JDBC connectors, for example) handle the dirty work. A few years ago, I wrote a load generation application using Kotlin’s coroutines - in this case, each coroutine would be a “device”. And I could add interesting modeling on each device; I easily ran 250k simulated devices within a single process, and it took me a couple of days. But coroutines are not totally simple; any method that might call IO needs to be made “coroutine aware”. So the abstraction kinda leaks all over the place. Now, you can do the same thing in Java. Just simply model each device as its own Runnable and poof, you can spin up a million of them. And there isn’t much existing code that has to be rewritten. Pretty slick. So this isn’t really a “high performance computing” feature, but a “blue collar coder” thing.
  - Svenskunganka 3y ago
    It's worth mentioning that there are some aspects of the virtual thread implementation that is important to take into consideration before switching out the os-thread-per-task executor for the virtual-thread-per-task executor: 1. Use of synchronized keyword can pin the virtual thread to the carrier thread, resulting in the carrier thread being blocked and unable to drive any other virtual threads. Recommendation is to refactor to use ReentrantLock. This may be solved in the future though. 2. Use of ThreadLocals should be reduced/avoided since they're local to the virtual thread and not the carrier threads, which could result in balooning memory usage with extensive usage from many virtual threads.
    - samus 3y ago
      Regarding the second point: there are now alternatives to ThreadLocals available that are intended to be used by virtual threads: Scoped Values. Unfortunately, they are preview-only in JDK 21.
- atomicnumber3 3y ago
  The main problem is that it's not a matter of "speed" but just of congestion. If you write a program using blocking IO and Platform (OS) threads, you're essentially limited to a couple hundred concurrent tasks, or however many threads your particular linux kernel + hardware setup can context switch between before latency starts suffering. So it's slow not because Java is slow, but because kernel threads are heavyweight and you can't just make a trillion of them just for them to be blocking waiting on IO. If you use async approaches, your programming model suffers, but now you're multiplexing millions of tasks over a small number of platform threads of execution still without even straining the kernel's switching. You've essentially moved from kernel scheduling to user-mode scheduling by writing async code. Virtual threads is a response to this, saying "what if you can eat your cake and have it, too?" by "simply" providing a user-mode-scheduled thread implementation. They took the general strategy that async programming was employing and "hoisted" it up a couple levels of abstraction, to be "behind" the threading model. Now you have all the benefits of just blocking the thread without all the problems that come from trying to have a ton of platform threads that will choke the linux kernel out.
zerr 3y ago
I remember they were talking about Green Threads 20 years ago. Is it something new?
- pjmlp 3y ago
  Yes, while green and red threads were an implementation detail not exposed to Java programmers, now both thread models are exposed (Threads and Virtual Threads). Additionally since virtual threads are exposed across the whole runtime and standard library, not only they are built on top of native threads (red), the developers have control over their scheduling.
- kaba0 3y ago
  Yes - green threads at the time were basically a solution to a hardware limitation. Virtual threads make blocking IO calls automagically non-blocking, allowing for better utilization of the CPU.
geodel 3y ago
Fully agree on Helidon Nima and blocking IO. Zero hope that Spring framework crapola will not smother already massively simplified thing with convenient abstractions on top it.
karg_kult 3y ago
Dotnet has had Tasks for years, seems like the same thing.
- sebazzz 3y ago
  Yes, but this is supposedly transparent. At least until you interface directly with native libraries, or with leaky abstractions that don’t account for that.
- 5e92cb50239222b 3y ago
  You're supposed to either await a Task, or to block on it (thus blocking the underlying OS thread which probably eats a couple of megabytes of RAM). It's a completely different system more akin to what Go has been using.
  - neonsunset 3y ago
    This is not necessarily correct. Tasks can be run in a "fire-and-forget" way. Also, only synchronous prelude of the task is executed inline in .NET. The continuation will then be ran on threadpool worker thread (unless you override task scheduler and continuation context). Also, you can create multiple tasks in a method and then await their results later down the execution path when you actually need it, easily achieving concurrency. Green threads is a more limited solution focused first and foremost on solving blocking.
    - kaba0 3y ago
      But there is no point to it in this comparison if it can block a whole thread to itself - that can be done with java as well since forever.
      - neonsunset 3y ago
        Blocking the thread with unawaited task.Result is an explicit choice which will be flagged by a warning by an IDE and when building, that this may not be what you intended.
PaulHoule 3y ago
Funny I think async in Python is a lot of fun for side projects but my experience is that if I hand my Python systems off to other people they usually have trouble deploying them and invariably can’t maintain them. Whereas my Java projects live on long after I am gone from the project. Personally I love aiohttp web servers, particularly when using web sockets and brokering events from message queues and stuff like that. Not to mention doing cool stuff with coroutines and hacking the event queue (like what do you do if your GUI framework also has an event queue?) If YOShInOn (my smart RSS reader + intelligent agent) were going to become open source though I might just need to switch to Flask which would be less fun.
- bad_user 3y ago
  Async in Python, for a long time, has been a horrible hack relying on monkey patching the socket module. The newer asyncio stuff is quite nice by comparison, but the problem is that Python, due to its popularity, has libraries that haven't been upgraded. Python always had deployment issues, IMO. In Java, 99% of all library dependencies are pure JARs, and you rarely need to depend on native libraries. You can also assemble an executable fat JAR which will work everywhere, and the fact that the build tools are better (e.g., Maven, Gradle) helps. Compared with Python, for which even accessing a RDBMS was an exercise in frustration, requiring installing the right blobs and library headers via the OS's package manager, with Postgres being particularly painful. NOTE: I haven't deployed anything serious built with Python in a while, maybe things are better now, but it couldn't get much better, IMO.
  - paulddraper 3y ago
    > In Java, 99% of all library dependencies are pure JARs Yes, this is the difference...Python community has in practice chosen more native dependencies, Java has not. But Java JNI code (if you ever do have it) is just as painful. > with Postgres being particularly painful You want to a pure Python package, and those have gotten much better. asyncpg is really, really good if you want async. Otherwise, pg8000.
    - 3y ago
davewritescode 3y ago
Debugging issues with Netty from someone not intimately familiar with it's internals was an exercise in pain.

eastbound 3y ago

And it’s LTS! When is the book and the certification exam coming out, does anyone know?

smallerfish 3y ago

Can you explain why you're excited about with virtual threads? I get that they improve throughput in extremely high pressure apps, but the JVM's current threading isn't exactly a slouch. Java's used in HFT shops, and more generally in fintech where performance matters.

mr_tristan 3y ago
I can’t speak for the OP, but this makes it much easier to write code that uses threads that wait on IO, and just let the underlying system (VM + JDBC connectors, for example) handle the dirty work. A few years ago, I wrote a load generation application using Kotlin’s coroutines - in this case, each coroutine would be a “device”. And I could add interesting modeling on each device; I easily ran 250k simulated devices within a single process, and it took me a couple of days. But coroutines are not totally simple; any method that might call IO needs to be made “coroutine aware”. So the abstraction kinda leaks all over the place. Now, you can do the same thing in Java. Just simply model each device as its own Runnable and poof, you can spin up a million of them. And there isn’t much existing code that has to be rewritten. Pretty slick. So this isn’t really a “high performance computing” feature, but a “blue collar coder” thing.
- Svenskunganka 3y ago
  It's worth mentioning that there are some aspects of the virtual thread implementation that is important to take into consideration before switching out the os-thread-per-task executor for the virtual-thread-per-task executor: 1. Use of synchronized keyword can pin the virtual thread to the carrier thread, resulting in the carrier thread being blocked and unable to drive any other virtual threads. Recommendation is to refactor to use ReentrantLock. This may be solved in the future though. 2. Use of ThreadLocals should be reduced/avoided since they're local to the virtual thread and not the carrier threads, which could result in balooning memory usage with extensive usage from many virtual threads.
  - samus 3y ago
    Regarding the second point: there are now alternatives to ThreadLocals available that are intended to be used by virtual threads: Scoped Values. Unfortunately, they are preview-only in JDK 21.
atomicnumber3 3y ago
The main problem is that it's not a matter of "speed" but just of congestion. If you write a program using blocking IO and Platform (OS) threads, you're essentially limited to a couple hundred concurrent tasks, or however many threads your particular linux kernel + hardware setup can context switch between before latency starts suffering. So it's slow not because Java is slow, but because kernel threads are heavyweight and you can't just make a trillion of them just for them to be blocking waiting on IO. If you use async approaches, your programming model suffers, but now you're multiplexing millions of tasks over a small number of platform threads of execution still without even straining the kernel's switching. You've essentially moved from kernel scheduling to user-mode scheduling by writing async code. Virtual threads is a response to this, saying "what if you can eat your cake and have it, too?" by "simply" providing a user-mode-scheduled thread implementation. They took the general strategy that async programming was employing and "hoisted" it up a couple levels of abstraction, to be "behind" the threading model. Now you have all the benefits of just blocking the thread without all the problems that come from trying to have a ton of platform threads that will choke the linux kernel out.

zerr 3y ago

I remember they were talking about Green Threads 20 years ago. Is it something new?

pjmlp 3y ago
Yes, while green and red threads were an implementation detail not exposed to Java programmers, now both thread models are exposed (Threads and Virtual Threads). Additionally since virtual threads are exposed across the whole runtime and standard library, not only they are built on top of native threads (red), the developers have control over their scheduling.
kaba0 3y ago
Yes - green threads at the time were basically a solution to a hardware limitation. Virtual threads make blocking IO calls automagically non-blocking, allowing for better utilization of the CPU.

geodel 3y ago

Fully agree on Helidon Nima and blocking IO. Zero hope that Spring framework crapola will not smother already massively simplified thing with convenient abstractions on top it.

karg_kult 3y ago

Dotnet has had Tasks for years, seems like the same thing.

sebazzz 3y ago
Yes, but this is supposedly transparent. At least until you interface directly with native libraries, or with leaky abstractions that don’t account for that.
5e92cb50239222b 3y ago
You're supposed to either await a Task, or to block on it (thus blocking the underlying OS thread which probably eats a couple of megabytes of RAM). It's a completely different system more akin to what Go has been using.
- neonsunset 3y ago
  This is not necessarily correct. Tasks can be run in a "fire-and-forget" way. Also, only synchronous prelude of the task is executed inline in .NET. The continuation will then be ran on threadpool worker thread (unless you override task scheduler and continuation context). Also, you can create multiple tasks in a method and then await their results later down the execution path when you actually need it, easily achieving concurrency. Green threads is a more limited solution focused first and foremost on solving blocking.
  - kaba0 3y ago
    But there is no point to it in this comparison if it can block a whole thread to itself - that can be done with java as well since forever.
    - neonsunset 3y ago
      Blocking the thread with unawaited task.Result is an explicit choice which will be flagged by a warning by an IDE and when building, that this may not be what you intended.

PaulHoule 3y ago

Funny I think async in Python is a lot of fun for side projects but my experience is that if I hand my Python systems off to other people they usually have trouble deploying them and invariably can’t maintain them. Whereas my Java projects live on long after I am gone from the project. Personally I love aiohttp web servers, particularly when using web sockets and brokering events from message queues and stuff like that. Not to mention doing cool stuff with coroutines and hacking the event queue (like what do you do if your GUI framework also has an event queue?) If YOShInOn (my smart RSS reader + intelligent agent) were going to become open source though I might just need to switch to Flask which would be less fun.

bad_user 3y ago
Async in Python, for a long time, has been a horrible hack relying on monkey patching the socket module. The newer asyncio stuff is quite nice by comparison, but the problem is that Python, due to its popularity, has libraries that haven't been upgraded. Python always had deployment issues, IMO. In Java, 99% of all library dependencies are pure JARs, and you rarely need to depend on native libraries. You can also assemble an executable fat JAR which will work everywhere, and the fact that the build tools are better (e.g., Maven, Gradle) helps. Compared with Python, for which even accessing a RDBMS was an exercise in frustration, requiring installing the right blobs and library headers via the OS's package manager, with Postgres being particularly painful. NOTE: I haven't deployed anything serious built with Python in a while, maybe things are better now, but it couldn't get much better, IMO.
- paulddraper 3y ago
  > In Java, 99% of all library dependencies are pure JARs Yes, this is the difference...Python community has in practice chosen more native dependencies, Java has not. But Java JNI code (if you ever do have it) is just as painful. > with Postgres being particularly painful You want to a pure Python package, and those have gotten much better. asyncpg is really, really good if you want async. Otherwise, pg8000.
  - 3y ago

davewritescode 3y ago

Debugging issues with Netty from someone not intimately familiar with it's internals was an exercise in pain.