10 ms·

Using Java's Project Loom to build more reliable distributed systems

205 points by CHY872 4y ago

_benedict 4y ago
Cassandra already does this[1] (and a whole lot more besides to introduce chaos to the simulation), and just accepts that there's a huge penalty of pausing and scheduling threads. I'm really looking forward to being able to use Loom to run much more efficient cluster simulations. [1] https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-10%3A+Cluster+and+Code+Simulations https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-10...
- CHY872 4y ago
  What sort of performance characteristics does Cassandra see from its implementation? I was fairly impressed with the 2.5M context switches per second (a fairly random unit, I know) I saw from virtual threads. Have you found that the approach has made it easier for you to ship improvements to Cassandra?
  - _benedict 4y ago
    It's very slow, particularly since introducing byte weaving to introduce chaos in concurrency-sensitive places. Presently it takes a single thread in the region of twenty minutes to simulate a cluster from startup to shutdown, and to run a thousand linearisable operations (down from a couple of minutes prior to those changes). I haven't got any numbers on the rate of context switches per second, but it is certainly a great deal less than 2.5M/s. This kind of approach is in my opinion a _requirement_ for work on distributed consensus protocols, which the Cassandra community is improving at the moment: the upcoming 4.1 release has some major improvements to its Paxos implementation that were validated by this simulation, and the following release is expected to contain a novel distributed consensus protocol we have named Accord, for supporting multi-key transactions. So, I'm not sure it has affected the speed or ease of shipping improvements, rather than enabled a class of work that would have previously been impossible to do safely (IMO).
exabrial 4y ago
This was linked in the article and is a fascinating read by itself: https://shipilev.net/blog/2016/close-encounters-of-jmm-kind https://shipilev.net/blog/2016/close-encounters-of-jmm-kind
- dang 4y ago
  Discussed at the time: Close Encounters of the Java Memory Model Kind - https://news.ycombinator.com/item?id=11955392 https://news.ycombinator.com/item?id=11955392 - June 2016 (65 comments)
- vips7L 4y ago
  Shipilev’s entire blog is worth reading. He’ll even email you back if you have questions. I really enjoy his anatomy quarks series: https://shipilev.net/jvm/anatomy-quarks/ https://shipilev.net/jvm/anatomy-quarks/
deleted 4y ago
[deleted]
tadfisher 4y ago
So Loom is just green threads? You have to implement a CPS transformation on top? If so, great decision!
- cogman10 4y ago
  Loom is just green threads (referred to as virtual threads).
  - heartofgold 4y ago
    For any older java programmers that may have a bad taste in their mouth when they think of green threads: when Java first came out, it had green threads, and all N threads that were spawned were tied to a single kernel thread. It also required cooperative multithreading, where one green thread may need to yield. Java upgraded to native threads, and then you could have N java threads bound to N kernel threads. This was way better, but had downsides: You're limited on how many threads you can spawn, you need a threadpool to help manage, and any long-running tasks could effectively deplete your thread-pool. With Loom, now you have M green threads mapped to N kernel threads. These green threads are way cheaper to spawn, so you could have thousands (millions even?) of green threads. Blocking calls won't tie up a kernel thread. So if you have many long-running IO tasks, they aren't going to waste a kernel thread and have it sit around idle waiting on IO. This is similar to async libraries, but without the mental overhead. You should be able to just code synchronously and the JVM will take care of the rest.
    - ackfoobar 4y ago
      > all N threads that were spawned were tied to a single kernel thread. It also required cooperative multithreading, where one green thread may need to yield. This is pretty much how JavaScript operates. You can consider calling an async function as spawning a user-level "thread"; chained-up callbacks are the same thing, but with manual CPS transform. This has always perplexed me. Why N:1 threading is hated, but Node.JS is so loved?
      - 4y ago
electricshampo1 4y ago
Unlike goroutines, seems here you have control over the execution schedule for the virtual threads if you provide an executor. This is pretty great. Think this will obsolete go over the next few decades.
- cogman10 4y ago
  Not on the first go around. AFAIK, they are looking at exposing more of the internal scheduling API but that's likely not going to be a part of the initial release. The executor services referred to in the blog are for the order of execution of tasks on the virtual thread pool. For a "virtualThreadExecutor" service, every task will get a virtual thread and scheduling will happen internally. You can still use a fixed thread pool with a custom task scheduler if you like, but probably not exactly what you are after.
  - CHY872 4y ago
    This blogpost does rely on plugging in an executor. While the API was removed, it’s one private variable away (documented in footnote). As you say, it seems like it’s an ‘on the way, but later’ thing - the last Loom preview I used (a while ago) actually had the API so when I started drafting this post I was unhappily surprised!
    - cogman10 4y ago
      IMO, they are focusing on the right thing here. Getting a good virtual thread API GA will be paramount in the decisions around scheduling and continuations in the future. Maybe a little disappointing for low level nuts and other languages like kotlin, but the right move IMO. Virtual threads alone will be a huge benefit to the ecosystem. The other stuff will help, but won't have near the same impact.
      - CHY872 4y ago
        Agreed. And the pluggable scheduling is still a key part of it, just doesn’t need to be a thing to get right out of the gate. I’m honestly mega excited that Loom is even a real thing that exists and that you can use.
theptip 4y ago
This is an interesting idea. I've been idly thinking about how to make a framework for detecting DB anomalies in Django applications (say, missing transactions, race conditions, deadlocks etc.) which can be hard to detect. You can strap your application to Jepsen but this is a more white-box approach and probably harder to grok for the average Python developer. I like the OP's idea of using a virtual thread implementation to parallelize the application-layer, while having the test code implement a custom executor to control the interleaving of the units-of-work being scheduled. I can see a few ways this approach could be used more widely; for example in Django you could write a DB driver wrapper and have this as the "test executor". Or for remote API requests, run the test code in green threads and then have a test executor that intercepts and then then chooses how to interleave the requests/responses.
zmj 4y ago
I once did something similar in C# to deterministically test some concurrency-heavy code (injecting custom TaskScheduler instances). It gave me a lot of confidence that it actually worked.
- rhplus 4y ago
  If you're looking for similar concurrency testing in the dotnet world, checkout Coyote: https://microsoft.github.io/coyote/ https://microsoft.github.io/coyote/ Explainer here: https://innovation.microsoft.com/en-us/exploring-project-coyote https://innovation.microsoft.com/en-us/exploring-project-coy...
kendallgclark 4y ago
The FoundationDb team is working on distributed systems simulation testing via a new startup... Antithesis. See https://antithesis.com/ https://antithesis.com/ Stardog, enterprise knowledge graph platform, (stardog.com), is an early adopter of Antithesis and we've found it to be very helpful in HA Cluster, distributed consensus, etc. Not a stakeholder, just a satisfied early adopter.
- ryanworl 4y ago
  This is the first reference to Antithesis being used I've seen since it was announced. Can you describe what their tools are doing for you and how you're integrating with them?
no_wizard 4y ago
I feel like the unsung winner of Project Loom is going to be Clojure. Its already immutable first data structures, it should be relatively straightforward for the Clojure project to expose the benefits of Project Loom to their ecosystem, as a language its designed to fit well its execution model.
- pvorb 4y ago
  Why should Clojure benefit more from Loom than other languages? I think it simplifies reactive/concurrent programming in any JVM language.
  - no_wizard 4y ago
    Same reason I always felt extremely comfortable with concurrent programming in F#. The immutable first principle makes it easier to reason about what the program is doing during concurrent execution. Of course all projects will benefit from Loom. I am merely positing that Clojure in particular could leverage this both most quickly and to very deep positive effect, due to the nature of the language itself putting immutability first, which if I recall correctly they built up a pattern of concurrent execution around already. Kotlin too will be be able to optimize their concurrency story. I simply wanted to posit an observation I hadn't seen elsewhere on the web about Project Loom yet.
  - eyelidlessness 4y ago
    Clojure was explicitly designed to make reasoning about concurrency simpler (most data is immutable and can be referenced in multiple routines without concern; mutable reference types use the same data semantics, wrapped in transactions) with minimal performance impact (immutable data types are implemented as persistent data structures, sharing memory rather than naive strategies like copy-on-write). I don’t know that it’ll necessarily benefit more from Loom than other JVM languages, so much as it’s well positioned to benefit from concurrency improvements generally because its underlying design encourages code well suited for taking advantage of concurrency generally. Edit to add: I hadn’t initially thought to speculate on the language itself taking advantage, but it occurred to me immediately after posting this comment. Another potential benefit is that many of its fundamental abstractions (most collection APIs are declarative) lend really well to making those operations concurrent, which is a common FP/lisp hypothetical but becomes more tangible if coroutines are less expensive. This isn’t dissimilar to how RDBMS query planners can provide wild performance improvements without any semantic changes, because SQL queries express “what” now “how”.
SemanticStrengh 4y ago
Will loom get support for Blockhound? https://github.com/reactor/BlockHound https://github.com/reactor/BlockHound Blockhound supports both reactive streams and Kotlin coroutines currently.
- delusional 4y ago
  Project loom is about deprecating "non-blocking" and "reactive" as a concept. The whole point is to just let the code block.
  - isbvhodnvemrwvn 4y ago
    How does it replace the backpressure mechanism from reactive solutions?
    - ackfoobar 4y ago
      Send to a full `BlockingQueue`, producer will block. Automatic backpressure.
rr808 4y ago
I spent the last few years really working hard to understand async frameworks/libraries like VertX, Mutiny etc I've gone from "I hate it" to "I can live with it". Now I dont have to. :) Yay.
qlm 4y ago
Any (relatively easy) way we can try virtual threads out yet?
- YawningAngel 4y ago
  You can download JDKs from https://jdk.java.net/loom/ https://jdk.java.net/loom/ that support it
- vips7L 4y ago
  It should be available in the 19 EA builds: https://jdk.java.net/19/ https://jdk.java.net/19/
yakorevivan 4y ago
deleted 4y ago
[deleted]

205 points by CHY872 4y ago

Cassandra already does this[1] (and a whole lot more besides to introduce chaos to the simulation), and just accepts that there's a huge penalty of pausing and scheduling threads. I'm really looking forward to being able to use Loom to run much more efficient cluster simulations. [1] https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-10%3A+Cluster+and+Code+Simulations https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-10...

CHY872 4y ago
What sort of performance characteristics does Cassandra see from its implementation? I was fairly impressed with the 2.5M context switches per second (a fairly random unit, I know) I saw from virtual threads. Have you found that the approach has made it easier for you to ship improvements to Cassandra?
- _benedict 4y ago
  It's very slow, particularly since introducing byte weaving to introduce chaos in concurrency-sensitive places. Presently it takes a single thread in the region of twenty minutes to simulate a cluster from startup to shutdown, and to run a thousand linearisable operations (down from a couple of minutes prior to those changes). I haven't got any numbers on the rate of context switches per second, but it is certainly a great deal less than 2.5M/s. This kind of approach is in my opinion a _requirement_ for work on distributed consensus protocols, which the Cassandra community is improving at the moment: the upcoming 4.1 release has some major improvements to its Paxos implementation that were validated by this simulation, and the following release is expected to contain a novel distributed consensus protocol we have named Accord, for supporting multi-key transactions. So, I'm not sure it has affected the speed or ease of shipping improvements, rather than enabled a class of work that would have previously been impossible to do safely (IMO).

exabrial 4y ago

This was linked in the article and is a fascinating read by itself: https://shipilev.net/blog/2016/close-encounters-of-jmm-kind https://shipilev.net/blog/2016/close-encounters-of-jmm-kind

dang 4y ago
Discussed at the time: Close Encounters of the Java Memory Model Kind - https://news.ycombinator.com/item?id=11955392 https://news.ycombinator.com/item?id=11955392 - June 2016 (65 comments)
vips7L 4y ago
Shipilev’s entire blog is worth reading. He’ll even email you back if you have questions. I really enjoy his anatomy quarks series: https://shipilev.net/jvm/anatomy-quarks/ https://shipilev.net/jvm/anatomy-quarks/

deleted 4y ago

[deleted]

tadfisher 4y ago

So Loom is just green threads? You have to implement a CPS transformation on top? If so, great decision!

cogman10 4y ago
Loom is just green threads (referred to as virtual threads).
- heartofgold 4y ago
  For any older java programmers that may have a bad taste in their mouth when they think of green threads: when Java first came out, it had green threads, and all N threads that were spawned were tied to a single kernel thread. It also required cooperative multithreading, where one green thread may need to yield. Java upgraded to native threads, and then you could have N java threads bound to N kernel threads. This was way better, but had downsides: You're limited on how many threads you can spawn, you need a threadpool to help manage, and any long-running tasks could effectively deplete your thread-pool. With Loom, now you have M green threads mapped to N kernel threads. These green threads are way cheaper to spawn, so you could have thousands (millions even?) of green threads. Blocking calls won't tie up a kernel thread. So if you have many long-running IO tasks, they aren't going to waste a kernel thread and have it sit around idle waiting on IO. This is similar to async libraries, but without the mental overhead. You should be able to just code synchronously and the JVM will take care of the rest.
  - ackfoobar 4y ago
    > all N threads that were spawned were tied to a single kernel thread. It also required cooperative multithreading, where one green thread may need to yield. This is pretty much how JavaScript operates. You can consider calling an async function as spawning a user-level "thread"; chained-up callbacks are the same thing, but with manual CPS transform. This has always perplexed me. Why N:1 threading is hated, but Node.JS is so loved?
    - 4y ago

electricshampo1 4y ago

Unlike goroutines, seems here you have control over the execution schedule for the virtual threads if you provide an executor. This is pretty great. Think this will obsolete go over the next few decades.

cogman10 4y ago
Not on the first go around. AFAIK, they are looking at exposing more of the internal scheduling API but that's likely not going to be a part of the initial release. The executor services referred to in the blog are for the order of execution of tasks on the virtual thread pool. For a "virtualThreadExecutor" service, every task will get a virtual thread and scheduling will happen internally. You can still use a fixed thread pool with a custom task scheduler if you like, but probably not exactly what you are after.
- CHY872 4y ago
  This blogpost does rely on plugging in an executor. While the API was removed, it’s one private variable away (documented in footnote). As you say, it seems like it’s an ‘on the way, but later’ thing - the last Loom preview I used (a while ago) actually had the API so when I started drafting this post I was unhappily surprised!
  - cogman10 4y ago
    IMO, they are focusing on the right thing here. Getting a good virtual thread API GA will be paramount in the decisions around scheduling and continuations in the future. Maybe a little disappointing for low level nuts and other languages like kotlin, but the right move IMO. Virtual threads alone will be a huge benefit to the ecosystem. The other stuff will help, but won't have near the same impact.
    - CHY872 4y ago
      Agreed. And the pluggable scheduling is still a key part of it, just doesn’t need to be a thing to get right out of the gate. I’m honestly mega excited that Loom is even a real thing that exists and that you can use.

theptip 4y ago

This is an interesting idea. I've been idly thinking about how to make a framework for detecting DB anomalies in Django applications (say, missing transactions, race conditions, deadlocks etc.) which can be hard to detect. You can strap your application to Jepsen but this is a more white-box approach and probably harder to grok for the average Python developer. I like the OP's idea of using a virtual thread implementation to parallelize the application-layer, while having the test code implement a custom executor to control the interleaving of the units-of-work being scheduled. I can see a few ways this approach could be used more widely; for example in Django you could write a DB driver wrapper and have this as the "test executor". Or for remote API requests, run the test code in green threads and then have a test executor that intercepts and then then chooses how to interleave the requests/responses.

zmj 4y ago

I once did something similar in C# to deterministically test some concurrency-heavy code (injecting custom TaskScheduler instances). It gave me a lot of confidence that it actually worked.

rhplus 4y ago
If you're looking for similar concurrency testing in the dotnet world, checkout Coyote: https://microsoft.github.io/coyote/ https://microsoft.github.io/coyote/ Explainer here: https://innovation.microsoft.com/en-us/exploring-project-coyote https://innovation.microsoft.com/en-us/exploring-project-coy...

kendallgclark 4y ago

The FoundationDb team is working on distributed systems simulation testing via a new startup... Antithesis. See https://antithesis.com/ https://antithesis.com/ Stardog, enterprise knowledge graph platform, (stardog.com), is an early adopter of Antithesis and we've found it to be very helpful in HA Cluster, distributed consensus, etc. Not a stakeholder, just a satisfied early adopter.

ryanworl 4y ago
This is the first reference to Antithesis being used I've seen since it was announced. Can you describe what their tools are doing for you and how you're integrating with them?

no_wizard 4y ago

I feel like the unsung winner of Project Loom is going to be Clojure. Its already immutable first data structures, it should be relatively straightforward for the Clojure project to expose the benefits of Project Loom to their ecosystem, as a language its designed to fit well its execution model.

pvorb 4y ago
Why should Clojure benefit more from Loom than other languages? I think it simplifies reactive/concurrent programming in any JVM language.
- no_wizard 4y ago
  Same reason I always felt extremely comfortable with concurrent programming in F#. The immutable first principle makes it easier to reason about what the program is doing during concurrent execution. Of course all projects will benefit from Loom. I am merely positing that Clojure in particular could leverage this both most quickly and to very deep positive effect, due to the nature of the language itself putting immutability first, which if I recall correctly they built up a pattern of concurrent execution around already. Kotlin too will be be able to optimize their concurrency story. I simply wanted to posit an observation I hadn't seen elsewhere on the web about Project Loom yet.
- eyelidlessness 4y ago
  Clojure was explicitly designed to make reasoning about concurrency simpler (most data is immutable and can be referenced in multiple routines without concern; mutable reference types use the same data semantics, wrapped in transactions) with minimal performance impact (immutable data types are implemented as persistent data structures, sharing memory rather than naive strategies like copy-on-write). I don’t know that it’ll necessarily benefit more from Loom than other JVM languages, so much as it’s well positioned to benefit from concurrency improvements generally because its underlying design encourages code well suited for taking advantage of concurrency generally. Edit to add: I hadn’t initially thought to speculate on the language itself taking advantage, but it occurred to me immediately after posting this comment. Another potential benefit is that many of its fundamental abstractions (most collection APIs are declarative) lend really well to making those operations concurrent, which is a common FP/lisp hypothetical but becomes more tangible if coroutines are less expensive. This isn’t dissimilar to how RDBMS query planners can provide wild performance improvements without any semantic changes, because SQL queries express “what” now “how”.

SemanticStrengh 4y ago

Will loom get support for Blockhound? https://github.com/reactor/BlockHound https://github.com/reactor/BlockHound Blockhound supports both reactive streams and Kotlin coroutines currently.

delusional 4y ago
Project loom is about deprecating "non-blocking" and "reactive" as a concept. The whole point is to just let the code block.
- isbvhodnvemrwvn 4y ago
  How does it replace the backpressure mechanism from reactive solutions?
  - ackfoobar 4y ago
    Send to a full `BlockingQueue`, producer will block. Automatic backpressure.

rr808 4y ago

I spent the last few years really working hard to understand async frameworks/libraries like VertX, Mutiny etc I've gone from "I hate it" to "I can live with it". Now I dont have to. :) Yay.

qlm 4y ago

Any (relatively easy) way we can try virtual threads out yet?

YawningAngel 4y ago
You can download JDKs from https://jdk.java.net/loom/ https://jdk.java.net/loom/ that support it
vips7L 4y ago
It should be available in the 19 EA builds: https://jdk.java.net/19/ https://jdk.java.net/19/

yakorevivan 4y ago

deleted 4y ago

[deleted]