7 ms·

The first release to contain bits for Project Loom (the "virtual threads" part of the release notes). That is extremely exciting, since it is designed in a way

The first release to contain bits for Project Loom (the "virtual threads" part of the release notes). That is extremely exciting, since it is designed in a way that mostly allows drop-in replacement of existing threading and thread pool code, without requiring an utterly different programming model like async, coroutines or reactive programming do.

likeabbas 4y ago
I wish more languages would have gone this route (looking at you, Rust)
- amaranth 4y ago
  Rust started on this route but decided easy C interop and no required runtime were more important and trying to have everything made green threads slower than native threads.
  - likeabbas 4y ago
    They wouldn’t have had to include a runtime. They just needed to set a standard interface for kernel threads and user threads and then the community could’ve built the runtimes in libraries like they’re currently doing. But they didn’t create those interfaces early enough so the community built them and now the async ecosystem is so fragmented you can’t build libraries generically for async runtimes or for both async and sync
    - pkolaczk 4y ago
      > is so fragmented you can’t build libraries generically for async runtimes How does futures.rs work generically with different async runtimes then?
      - likeabbas 4y ago
        It doesn't work for all async runtimes like Monoio
    - amaranth 4y ago
      A different runtime wouldn't be able to make the compiler use a different stack allocation strategy (like Go's segmented stacks), for that the compiler needs to know what you're doing. Even if they also did that via having two ABIs for every platform (green vs native) it would essentially bake in a particular runtime as you wouldn't be able to experiment with these lower level primitives, only how you interface with them. If they ever define a stable ABI having two ABIs depending on thread type would also fragment the ecosystem at least as bad as the async runtimes do. As far as async runtime fragmentation you can write code that is generic across runtimes, it's just less convenient to use. If your code only works with plain data and futures you're fine on any runtime, it's when you want to spawn new tasks or block on a task that you have to have a runtime dependency. Async vs sync is the classic "what color is your function" problem which green threads kind of solves but not completely so you still have to understand what is happening so you don't stall every task on a native thread with CPU heavy or blocking work.
    - saghm 4y ago
      > now the async ecosystem is so fragmented you can’t build libraries generically for async runtimes or for both async and sync This is true, but I think there are a few notable caveats: 1. Although you can't generically build libraries per runtime, it is possible to right a library supporting an explicit set of runtimes with some boilerplate; the simplest way is to just to abstract all of the async primitives you want into an API to use internally, use feature flags to implement them based on which runtime you want to support, and then only use that API in the rest of your library (which both avoids the need for conditional compilation outside of that one wrapper module for async stuff and reduces the surface of places you'd need to update to add or remove support for a runtime or if you want to update a runtime to a version that isn't backwards compatible. 2. Although there are quite a lot of runtimes that exist in the ecosystem right now (and more could enter the scene in the future), in practice the usage of a lot of these runtimes is quite small. For a few examples, at the time of my looking up, tokio has 66.7 million downloads overall and 10.3 million "recent" ones; async-std has 9 million overall and 1.3 million recent; smol has 1.6 million overall and 158k recent. There's diminishing returns for each new runtime you add support for, so while there's some up front burden in terms of supporting more than one runtime (like I describe above), the burden over time is not going to be super high. 3. Rust has a history of starting out by providing lower-level primitives for things, letting the community iterate over various ideas in the space for a few years, and then eventually settling on a single or small set of pseudo-official crates for them. I think the error handling APIs are the best example of this; Rust 1.0 launched with the standard library Error trait and the `try!` macro, and the community iterated over a bunch of potential solution (error-chain, failure, and probably a bunch of others I don't even remember at the moment), and for a few years it was a bit messy. Meanwhile, the standard library added the `?` operator and added a replacement for one of the `Error` trait methods (deprecating the old ones), and eventually the churn settled down and most people just use `thiserror` and `anyhow` now. There's still some lingering things to standardize like backtraces for errors, but overall the error handling space is way less fragmented now than basically at any point in the past. I'd argue that the async runtime churn is already starting to trend towards equilibrium; if this turns out to be the case, the costs of supporting multiple runtimes will continue to go down, and I'm guessing (hoping?) that within a few years people will have either mostly standardized on a single runtime or we'll have a standard solution for wrapping the small set of runtimes that retain any significant usage in the community.
- kaba0 4y ago
  It’s not really possible in the niche rust plays at. If you want to remain close to low-level detail. you have to deal with low-level details.
jillesvangurp 4y ago
Virtual threads, real threads, and structured concurrency is quite literally what co-routines are about. From a Kotlin co-routine point of view Loom is about the JVM gaining some low level optimization features that will slightly enhance co-routine performance on newer JDKs (not that this was actually much of a problem) but will otherwise not really add anything new in terms of features as co-routines already had pretty much all of the features. Of course it's nice for Java to gain a few new APIs for this. But it's not like there was a shortage of asynchronous programming APIs. If you intend to use this as a drop in replacement for Threads, read up on blocking vs. non blocking IO or you'll be finding out the hard way why Java uses a lot of real threads when dealing with blocking IO. If on the other hand you were already using non blocking IO, using Threads would not have been a thing you'd be doing. Hint, virtual threads in the same thread will all block if you use blocking IO. With Kotlin, co-routines, the compiler will issue warnings if you call into blocking Java stuff from a co-routine and nudge you to deal with that by of-loading to a threaded co-routine context.
- mwcampbell 4y ago
  > Hint, virtual threads in the same thread will all block if you use blocking IO. This is apparently not the case in the new JDK implementation of virtual threads. According to JEP 425 (linked in the OP): > Application code in the thread-per-request style can run in a virtual thread for the entire duration of a request, but the virtual thread consumes an OS thread only while it performs calculations on the CPU. The result is the same scalability as the asynchronous style, except it is achieved transparently: When code running in a virtual thread calls a blocking I/O operation in the java.* API, the runtime performs a non-blocking OS call and automatically suspends the virtual thread until it can be resumed later. To Java developers, virtual threads are simply threads that are cheap to create and almost infinitely plentiful. Hardware utilization is close to optimal, allowing a high level of concurrency and, as a result, high throughput, while the application remains harmonious with the multithreaded design of the Java Platform and its tooling. So new Java virtual threads really are meaningfully different from Kotlin coroutines.
  - jillesvangurp 4y ago
djhworld 4y ago
In the JEP they reference go and erlang as other examples of successful runtimes adopting this approach. Which is true around the M:N stuff, but the missing piece is the communication. Go has channels, Erlang uses the actor model, do you know if there are any plans to have something like this in Java?
- kaba0 4y ago
  Go has those inbuilt because before generics the language simply wasn’t expressive enough for such constructs.
- Skinney 4y ago
  You already have queues in Java. They should work just fine with virtual threads, as they do on regular threads.
- pjmlp 4y ago
  Java already has better ways to do that on java.util.concurrent, and structured concurrency is coming as well.
  - PaulHoule 4y ago
    I wasn't too impressed when I read the docs on structured concurrency but they might flesh them out. Frequently I've written applications that use Executors, Reactive Streams, etc. and found that frameworks like that rarely, if ever, provide a good answer for how to tear a processing system down at the end. It's completely practical to use a stream processing framework to process batch jobs, but to get correct answers you have to shut the thing down at the end. I've frequently built teardown systems and it's always left me with the feeling that system programmers should occasionally climb down from their high horse and write an application.
    - Skinney 4y ago
      > but to get correct answers you have to shut the thing down at the end That's one of the things structured concurrency improves on. By defining a scope in a try-with-resources block, you make sure it's teared down at the end of whatever operation you perform.
jayd16 4y ago
Virtual threads are nice but they don't cover the same use cases as async and coroutines. Loom will make embarrassingly parallel workloads seamless but it won't give you structured concurrency. That does require a different programming model as seen in this very release. Should be interesting to see what shakes out of all this. When you need structured concurrency, imo async and coroutines are nicer than what fork/join and what Java has so far.
- mping 4y ago
  As I understand it, structured concurrency is part of the same deliverable, tracked in this JEP: https://openjdk.org/jeps/428 https://openjdk.org/jeps/428 Here's the example from the JEP: Response handle() throws ExecutionException, InterruptedException { try (var scope = new StructuredTaskScope.ShutdownOnFailure()) { Future<String> user = scope.fork(() -> findUser()); Future<Integer> order = scope.fork(() -> fetchOrder()); scope.join(); // Join both forks scope.throwIfFailed(); // ... and propagate errors // Here, both forks have succeeded, so compose their results return new Response(user.resultNow(), order.resultNow()); } }
  - jayd16 4y ago
    I'm aware of the JEP. I said "as seen in this very release" after all. I don't see the advantage to this over async/await style programming. I think you'll see very similar error cases with unobserved exceptions and such. The advantage is that the method signature is indistinguishable from a blocking method. The subtle difference is that scopes close within a method where as async tasks can continue. I suppose you could see an even worse coloring where scopes and futures are used in half the methods instead of async. I'm trying to wrap my head around how this would play out if you were attempting to write a GUI or something where the pattern is a single main thread. I suppose it would work just fine and look very similar to async style programming but with a lot more code one tab to the right.
    - didibus 4y ago
mwcampbell 4y ago
Now that blocking APIs are no longer a problem, I wonder if green-field Java web backend projects should use an HTTP server based on Netty with a blocking layer on top, a servlet implementation like Jetty that was designed for blocking APIs but might have other legacy baggage, or something else.
- kaba0 4y ago
  Helidon Nima is a new contender, made with virtual threads in mind.

likeabbas 4y ago

I wish more languages would have gone this route (looking at you, Rust)

amaranth 4y ago
Rust started on this route but decided easy C interop and no required runtime were more important and trying to have everything made green threads slower than native threads.
- likeabbas 4y ago
  They wouldn’t have had to include a runtime. They just needed to set a standard interface for kernel threads and user threads and then the community could’ve built the runtimes in libraries like they’re currently doing. But they didn’t create those interfaces early enough so the community built them and now the async ecosystem is so fragmented you can’t build libraries generically for async runtimes or for both async and sync
  - pkolaczk 4y ago
    > is so fragmented you can’t build libraries generically for async runtimes How does futures.rs work generically with different async runtimes then?
    - likeabbas 4y ago
      It doesn't work for all async runtimes like Monoio
  - amaranth 4y ago
    A different runtime wouldn't be able to make the compiler use a different stack allocation strategy (like Go's segmented stacks), for that the compiler needs to know what you're doing. Even if they also did that via having two ABIs for every platform (green vs native) it would essentially bake in a particular runtime as you wouldn't be able to experiment with these lower level primitives, only how you interface with them. If they ever define a stable ABI having two ABIs depending on thread type would also fragment the ecosystem at least as bad as the async runtimes do. As far as async runtime fragmentation you can write code that is generic across runtimes, it's just less convenient to use. If your code only works with plain data and futures you're fine on any runtime, it's when you want to spawn new tasks or block on a task that you have to have a runtime dependency. Async vs sync is the classic "what color is your function" problem which green threads kind of solves but not completely so you still have to understand what is happening so you don't stall every task on a native thread with CPU heavy or blocking work.
  - saghm 4y ago
    > now the async ecosystem is so fragmented you can’t build libraries generically for async runtimes or for both async and sync This is true, but I think there are a few notable caveats: 1. Although you can't generically build libraries per runtime, it is possible to right a library supporting an explicit set of runtimes with some boilerplate; the simplest way is to just to abstract all of the async primitives you want into an API to use internally, use feature flags to implement them based on which runtime you want to support, and then only use that API in the rest of your library (which both avoids the need for conditional compilation outside of that one wrapper module for async stuff and reduces the surface of places you'd need to update to add or remove support for a runtime or if you want to update a runtime to a version that isn't backwards compatible. 2. Although there are quite a lot of runtimes that exist in the ecosystem right now (and more could enter the scene in the future), in practice the usage of a lot of these runtimes is quite small. For a few examples, at the time of my looking up, tokio has 66.7 million downloads overall and 10.3 million "recent" ones; async-std has 9 million overall and 1.3 million recent; smol has 1.6 million overall and 158k recent. There's diminishing returns for each new runtime you add support for, so while there's some up front burden in terms of supporting more than one runtime (like I describe above), the burden over time is not going to be super high. 3. Rust has a history of starting out by providing lower-level primitives for things, letting the community iterate over various ideas in the space for a few years, and then eventually settling on a single or small set of pseudo-official crates for them. I think the error handling APIs are the best example of this; Rust 1.0 launched with the standard library Error trait and the `try!` macro, and the community iterated over a bunch of potential solution (error-chain, failure, and probably a bunch of others I don't even remember at the moment), and for a few years it was a bit messy. Meanwhile, the standard library added the `?` operator and added a replacement for one of the `Error` trait methods (deprecating the old ones), and eventually the churn settled down and most people just use `thiserror` and `anyhow` now. There's still some lingering things to standardize like backtraces for errors, but overall the error handling space is way less fragmented now than basically at any point in the past. I'd argue that the async runtime churn is already starting to trend towards equilibrium; if this turns out to be the case, the costs of supporting multiple runtimes will continue to go down, and I'm guessing (hoping?) that within a few years people will have either mostly standardized on a single runtime or we'll have a standard solution for wrapping the small set of runtimes that retain any significant usage in the community.
kaba0 4y ago
It’s not really possible in the niche rust plays at. If you want to remain close to low-level detail. you have to deal with low-level details.

jillesvangurp 4y ago

Virtual threads, real threads, and structured concurrency is quite literally what co-routines are about. From a Kotlin co-routine point of view Loom is about the JVM gaining some low level optimization features that will slightly enhance co-routine performance on newer JDKs (not that this was actually much of a problem) but will otherwise not really add anything new in terms of features as co-routines already had pretty much all of the features. Of course it's nice for Java to gain a few new APIs for this. But it's not like there was a shortage of asynchronous programming APIs. If you intend to use this as a drop in replacement for Threads, read up on blocking vs. non blocking IO or you'll be finding out the hard way why Java uses a lot of real threads when dealing with blocking IO. If on the other hand you were already using non blocking IO, using Threads would not have been a thing you'd be doing. Hint, virtual threads in the same thread will all block if you use blocking IO. With Kotlin, co-routines, the compiler will issue warnings if you call into blocking Java stuff from a co-routine and nudge you to deal with that by of-loading to a threaded co-routine context.

mwcampbell 4y ago
> Hint, virtual threads in the same thread will all block if you use blocking IO. This is apparently not the case in the new JDK implementation of virtual threads. According to JEP 425 (linked in the OP): > Application code in the thread-per-request style can run in a virtual thread for the entire duration of a request, but the virtual thread consumes an OS thread only while it performs calculations on the CPU. The result is the same scalability as the asynchronous style, except it is achieved transparently: When code running in a virtual thread calls a blocking I/O operation in the java.* API, the runtime performs a non-blocking OS call and automatically suspends the virtual thread until it can be resumed later. To Java developers, virtual threads are simply threads that are cheap to create and almost infinitely plentiful. Hardware utilization is close to optimal, allowing a high level of concurrency and, as a result, high throughput, while the application remains harmonious with the multithreaded design of the Java Platform and its tooling. So new Java virtual threads really are meaningfully different from Kotlin coroutines.
- jillesvangurp 4y ago

djhworld 4y ago

In the JEP they reference go and erlang as other examples of successful runtimes adopting this approach. Which is true around the M:N stuff, but the missing piece is the communication. Go has channels, Erlang uses the actor model, do you know if there are any plans to have something like this in Java?

kaba0 4y ago
Go has those inbuilt because before generics the language simply wasn’t expressive enough for such constructs.
Skinney 4y ago
You already have queues in Java. They should work just fine with virtual threads, as they do on regular threads.
pjmlp 4y ago
Java already has better ways to do that on java.util.concurrent, and structured concurrency is coming as well.
- PaulHoule 4y ago
  I wasn't too impressed when I read the docs on structured concurrency but they might flesh them out. Frequently I've written applications that use Executors, Reactive Streams, etc. and found that frameworks like that rarely, if ever, provide a good answer for how to tear a processing system down at the end. It's completely practical to use a stream processing framework to process batch jobs, but to get correct answers you have to shut the thing down at the end. I've frequently built teardown systems and it's always left me with the feeling that system programmers should occasionally climb down from their high horse and write an application.
  - Skinney 4y ago
    > but to get correct answers you have to shut the thing down at the end That's one of the things structured concurrency improves on. By defining a scope in a try-with-resources block, you make sure it's teared down at the end of whatever operation you perform.

jayd16 4y ago

Virtual threads are nice but they don't cover the same use cases as async and coroutines. Loom will make embarrassingly parallel workloads seamless but it won't give you structured concurrency. That does require a different programming model as seen in this very release. Should be interesting to see what shakes out of all this. When you need structured concurrency, imo async and coroutines are nicer than what fork/join and what Java has so far.

mping 4y ago
As I understand it, structured concurrency is part of the same deliverable, tracked in this JEP: https://openjdk.org/jeps/428 https://openjdk.org/jeps/428 Here's the example from the JEP: Response handle() throws ExecutionException, InterruptedException { try (var scope = new StructuredTaskScope.ShutdownOnFailure()) { Future<String> user = scope.fork(() -> findUser()); Future<Integer> order = scope.fork(() -> fetchOrder()); scope.join(); // Join both forks scope.throwIfFailed(); // ... and propagate errors // Here, both forks have succeeded, so compose their results return new Response(user.resultNow(), order.resultNow()); } }
- jayd16 4y ago
  I'm aware of the JEP. I said "as seen in this very release" after all. I don't see the advantage to this over async/await style programming. I think you'll see very similar error cases with unobserved exceptions and such. The advantage is that the method signature is indistinguishable from a blocking method. The subtle difference is that scopes close within a method where as async tasks can continue. I suppose you could see an even worse coloring where scopes and futures are used in half the methods instead of async. I'm trying to wrap my head around how this would play out if you were attempting to write a GUI or something where the pattern is a single main thread. I suppose it would work just fine and look very similar to async style programming but with a lot more code one tab to the right.
  - didibus 4y ago

mwcampbell 4y ago

Now that blocking APIs are no longer a problem, I wonder if green-field Java web backend projects should use an HTTP server based on Netty with a blocking layer on top, a servlet implementation like Jetty that was designed for blocking APIs but might have other legacy baggage, or something else.

kaba0 4y ago
Helidon Nima is a new contender, made with virtual threads in mind.