12 ms·

Java 21 VirtualThreads vs. Clojure Lazy Seqs

215 points by grzm 3y ago

mike_hearn 3y ago
The Clojure devs should be aware that the synchronization pinning problem is intended to be temporary and doesn't apply at all with GraalVM native images. One of the Loom developers talks about it here: https://mail.openjdk.org/pipermail/loom-dev/2023-September/006178.html https://mail.openjdk.org/pipermail/loom-dev/2023-September/0... So they could also just choose to ignore this problem for now.
- deleted 3y ago
  [deleted]
- fnordsensei 3y ago
  A lot of Clojure projects choose to stick with the LTS releases though, so it might take a while before this problem disappears on its own, even if fixed in the JVM.
  - vemv 3y ago
    To be entirely fair, there's arguably little overlap between devs that stick to LTS, and devs that will play with JVM virtual threads on Clojure (before the Clojure team has any particular, public stance on them).
    - fulafel 3y ago
      Yes, especially since the use case for virtual threads is so niche.
      - weatherlight 3y ago
        Is this sarcasm, or just generally true in the java community?
        fulafel 3y ago
        Neither for me (I'm in the Clojure camp), it just seems to me there aren't many situations where you'd need millions of threads. Plus we've got core.async to reach for to multiplex many control flows on one OS thread - though I feel it's more needed when targeting JS (ClojureScript) and the single-threaded world there.
        weatherlight 3y ago
        I'm coming at this from a Erlang/Elixir background , Where we basically have "virtual threads" but they aren't at the OS level. It's easy to use and making things concurrent (or parallel) is often trivial. Whether it's 1 extra virtual thread or millions of virtual threads, the code (and the horizontal scaling of that code is the same)
        deleted 3y ago
        [deleted]
        raspasov 3y ago
        I love core.async (been using it since... 2014 I think... both Clojure and ClojureScript), and yes, core.async/go basically was/is solving (some) of the same problems. But there's a number of gotchas around (go ...) blocks... Without being an expert on the core.async internals, I believe core.async can potentially benefit tremendously from this, by the virtue of being a powerful and super-elegant API for communicating between different parts of an application (typically within the same JVM process). Now it can continue to do that, while (likely) being free from most macro gotchas...
        bcrosby95 3y ago
        I can't think of a project I've worked on in the past 10 or so years that couldn't benefit from virtual threads. Right-sizing your thread pools in an environment with a lot of IO is/was a pain in the ass. Many of the core concurrency constructs in Clojure has separate functions for when you are doing something that is blocking vs not blocking. If it were all virtual threads this distinction generally wouldn't matter.
      - deleted 3y ago
        [deleted]
      - raspasov 3y ago
        It's niche if you don't have much traffic/request volume/etc. Once you have even an arguably relatively small amount (100s to 1000s of requests per second) it can be a game changer in terms of efficiency.
        fulafel 3y ago
        Are there published case studies or other empirical evidence available with numbers about this? OS threads in Linux are fast and you can have a lot of them. Eg https://news.ycombinator.com/item?id=37621887 https://news.ycombinator.com/item?id=37621887
        raspasov 3y ago
        Just try starting ~100,000 threads that each sleeps for ~60 seconds in your JVM and check if you can succeed! :) ``` (run! (fn [x] (future (Thread/sleep 60000))) (range 100000)) ``` (hint: likely not...) TLDR; Efficiency difference: you can think of virtual threads as being about 1,000 times (3 orders of magnitude) less expensive, in the general case. Exception: If you are only doing CPU-only work, regular threads will be better (that's not how most web servers/services operate). But if you're waiting 100ms of ms for your database (or any network) to respond and you have many of those (blocking) method/function calls in flight... virtual threads are the way to go in terms of efficiency. Great video explaining all of this (at timestamp, all 30 min is worth watching): 17:05 Making threads less expensive: by how much? https://www.youtube.com/watch?v=5E0LU85EnTI&t=1025s https://www.youtube.com/watch?v=5E0LU85EnTI&t=1025s
        fulafel 3y ago
        On a wimpy old desktop running lots of other stuff I got 75000 threads with that snippet (had to increase max_map_count tunable first with sysctl -w vm.max_map_count=500000, a knob well documented for bigger thread counts). Considering that in a real world use case with that much concurrency (such as the "100k threads frequently waiting for 100ms db queries" scenario), I'd be using a bigger machine and there'd also be some actual application context data and TCP connection state dwarfing the thread memory requirements in those blocked contexts, I'll still call virtual threads solving quite rare use cases, pending stronger evidence. I don't doubt virtual threads are efficient when microbenchmarked against OS threads, but there's only so much to be gained when optimizing a part of the system that wasn't a bottleneck to begin with. (Also I hope in the scenario we have a beefy DB that can exploit the concurrency available in this many pending concurrent queries per client, and isn't just putting them in ever queue! I guess it could, maybe it has 1000 read replica servers, each with 100 attached SSDs or 100 cores serving from in-memory data)
  - vips7L 3y ago
    As pron always says.. that's a risk of staying on an LTS release. You're choosing to not receive updates.
    - geodel 3y ago
      It seems to me by design or by indifference Java is now tired of JVM ecosystem languages at large who claim innovation by surface level changes while relying heavily on JVM bedrock. Now Java is moving fast and they are taking JVM compiler, runtime, class metadata etc along where Java, the language is headed.
      - pjmlp 3y ago
        Not only Java, that is the fate of all guest languages, as they don't evolve alongside the platform, always require additional tooling and most language cultures than start making their own little islands of idiomatic libraries instead of directly using platform libraries. Since most platforms are leaky, one ends up needing to master two ecosystem in parallel, eventually it gets tiring and always cost extra in development. C++, Objective-C and Typescript get around this problem in UNIX and Web, as they are extensions of the platform languages, not something completely different.
        geodel 3y ago
        Yep agree with all you said. Nowadays new job every year people at work are scaring me with their enthusiasm to replace decades old solid Java applications which are performant and working fine with Kotlin and what not.
        Alupis 3y ago
        I wouldn't advocate for replacing/re-writing old stuff - but you should try out Kotlin for backend work. It's amazing. Java programmers will have only a small learning curve before feeling productive, and can gradually get more comfortable with idomatic Kotlin as they progress through their journey. Kotlin, as a language, was very well through out, letting you use as much or as little of it as you are comfortable with.
  - koito17 3y ago
    In fact, the officially supported Java releases for Clojure are all LTS releases. This is also reflected in their CI. At the moment, that means 1.8, 11, 17, and 21. I would also like to mention, for non-Clojure users: it took ages for Clojure to finally get off Java 1.5. The compiler was still emitting 1.5 compatible bytecode as late as 2018. Since the release of Clojure 1.10 (17 December 2018), the minimum Java version was bumped up to 1.8, in the sense that the compiler now emits 1.8-compatible bytecode. There is a strong emphasis on moving slowly and NOT breaking things in the Clojure community. Likewise, if you look at the most recent Clojure survey results, the vast majority of people were still on 1.8 and 11 releases.[1] [1] See Q24 in https://www.surveymonkey.com/stories/SM-_2BH3b49f_2FXEkUlrb_2BJSThxg_3D_3D/ https://www.surveymonkey.com/stories/SM-_2BH3b49f_2FXEkUlrb_...
    - tombert 3y ago
      I actually do feel like this is partly what's responsible for Clojure's relative success in industry. If you have a company with a ton of Java 5 code, and you want some of the sexy functional programming features, your options basically boil down to "upgrade everything to the newer Java 8 to get maps and optionals and whatnot" or "just write this new service in Clojure that will happily interop with our Java 5 code". I'm not making a judgement on which decision is better, but I can totally see the appeal, at least in the short term, for people choosing Clojure, especially if they think that Clojure's API will be largely unchanged in the Java 8 update. It doesn't hurt that Clojure is just broadly a really pleasant to work with, at least in my opinion.
      - Quekid5 3y ago
        > If you have a company with a ton of Java 5 code... You are in deep deep trouble already, just process-wise. If you're making money hand over fist you might be able to afford to do that, but choice of language is not even on the radar in terms of what needs fixing. Now, if you mean in terms of digging out of that hole... just upgrade to newer dependencies, newer JDK versions, etc. That's the way to better performance, better memory usage, better GC, better everything... and THEN think about the language you're using.
- puredanger 3y ago
  We are aware, and that is one of the options. :)
  - puredanger 3y ago
    The other thing I didn't mention is that the change to remove biased locking a while back has also had an impact in some places that do almost always uncontended locking, so we're kind of considering that too.
theanonymousone 3y ago
Still waiting for VS Code to support Java 21. Java is not the language to use without an IDE.
- jgon 3y ago
  Brian Goetz just gave a talk about all the stuff that's been delivered up to Java 21 that announced they'd be rolling out a fully supported VS Code plugin for Java that should be better than the current language support. It should be out within the next week or two. Timestamp for the announcement: https://youtu.be/eXCx2hW_xNI?t=1955 https://youtu.be/eXCx2hW_xNI?t=1955
- vips7L 3y ago
  Things take time. If you want you can contribute resources to the vs code plugin and the eclipse language server.
- avodonosov 3y ago
  What in the java language makes you think it is less suitable to be used without an IDE than other languages?
  - matsemann 3y ago
    It used to be lots of boilerplate. Like MyClassType myClassType = new MyClassType() And then it was nice to use an IDE to help write some of that. Or generate java beans etc. But with that said, I don't think it's much you need an IDE for, it's more that java enables IDEs to do much more than it can for certain other languages.
    - vips7L 3y ago
      You haven't had to declare the type since Java 10. var myThing = new MyThing();
      - riku_iki 3y ago
        still need it for class members, unlike in some other languages.
avodonosov 3y ago
Strange that synchnronization is used at all in lazy-seq The docs do not mention any synchronization: https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/lazy-seq https://clojure.github.io/clojure/clojure.core-api.html#cloj...
- lcedp 3y ago
  > a Seqable object that will invoke the body only the first time seq is called, and will cache the result and return it on all subsequent seq calls. To guarantee "only the first time", I suppose you need to have some kind of synchronization at some level.
  - avodonosov 3y ago
    For multithreaded access yes. I meant that the docs do not mention thread safety.
    - daveliepmann 3y ago
      "All of the Clojure collections...are efficient and inherently thread-safe." https://clojure.org/reference/data_structures#Collections https://clojure.org/reference/data_structures#Collections "Seqs differ from iterators in that they are persistent and immutable, not stateful cursors into a collection. As such, they are useful for much more than foreach - functions can consume and produce seqs, they are thread safe, they can share structure etc." https://clojure.org/reference/sequences https://clojure.org/reference/sequences I think it's a fundamental enough property of clojure data structures that it would be redundant to mention it in every docstring.
      - avodonosov 3y ago
        Thanks, that's convincing. (why I was in doubt, is because while immutable functional collections that clojure embaces are naturally thread-safe, there is an imperative aspect in creation of a lazy sequence from user provided function, and I was not sure clojure has to provide synchronization guarantees here)

215 points by grzm 3y ago

The Clojure devs should be aware that the synchronization pinning problem is intended to be temporary and doesn't apply at all with GraalVM native images. One of the Loom developers talks about it here: https://mail.openjdk.org/pipermail/loom-dev/2023-September/006178.html https://mail.openjdk.org/pipermail/loom-dev/2023-September/0... So they could also just choose to ignore this problem for now.

deleted 3y ago
[deleted]
fnordsensei 3y ago
A lot of Clojure projects choose to stick with the LTS releases though, so it might take a while before this problem disappears on its own, even if fixed in the JVM.
- vemv 3y ago
  To be entirely fair, there's arguably little overlap between devs that stick to LTS, and devs that will play with JVM virtual threads on Clojure (before the Clojure team has any particular, public stance on them).
  - fulafel 3y ago
    Yes, especially since the use case for virtual threads is so niche.
    - weatherlight 3y ago
      Is this sarcasm, or just generally true in the java community?
      - fulafel 3y ago
        Neither for me (I'm in the Clojure camp), it just seems to me there aren't many situations where you'd need millions of threads. Plus we've got core.async to reach for to multiplex many control flows on one OS thread - though I feel it's more needed when targeting JS (ClojureScript) and the single-threaded world there.
        weatherlight 3y ago
        I'm coming at this from a Erlang/Elixir background , Where we basically have "virtual threads" but they aren't at the OS level. It's easy to use and making things concurrent (or parallel) is often trivial. Whether it's 1 extra virtual thread or millions of virtual threads, the code (and the horizontal scaling of that code is the same)
        deleted 3y ago
        [deleted]
        raspasov 3y ago
        I love core.async (been using it since... 2014 I think... both Clojure and ClojureScript), and yes, core.async/go basically was/is solving (some) of the same problems. But there's a number of gotchas around (go ...) blocks... Without being an expert on the core.async internals, I believe core.async can potentially benefit tremendously from this, by the virtue of being a powerful and super-elegant API for communicating between different parts of an application (typically within the same JVM process). Now it can continue to do that, while (likely) being free from most macro gotchas...
      - bcrosby95 3y ago
        I can't think of a project I've worked on in the past 10 or so years that couldn't benefit from virtual threads. Right-sizing your thread pools in an environment with a lot of IO is/was a pain in the ass. Many of the core concurrency constructs in Clojure has separate functions for when you are doing something that is blocking vs not blocking. If it were all virtual threads this distinction generally wouldn't matter.
    - deleted 3y ago
      [deleted]
    - raspasov 3y ago
      It's niche if you don't have much traffic/request volume/etc. Once you have even an arguably relatively small amount (100s to 1000s of requests per second) it can be a game changer in terms of efficiency.
      - fulafel 3y ago
        Are there published case studies or other empirical evidence available with numbers about this? OS threads in Linux are fast and you can have a lot of them. Eg https://news.ycombinator.com/item?id=37621887 https://news.ycombinator.com/item?id=37621887
        raspasov 3y ago
        Just try starting ~100,000 threads that each sleeps for ~60 seconds in your JVM and check if you can succeed! :) ``` (run! (fn [x] (future (Thread/sleep 60000))) (range 100000)) ``` (hint: likely not...) TLDR; Efficiency difference: you can think of virtual threads as being about 1,000 times (3 orders of magnitude) less expensive, in the general case. Exception: If you are only doing CPU-only work, regular threads will be better (that's not how most web servers/services operate). But if you're waiting 100ms of ms for your database (or any network) to respond and you have many of those (blocking) method/function calls in flight... virtual threads are the way to go in terms of efficiency. Great video explaining all of this (at timestamp, all 30 min is worth watching): 17:05 Making threads less expensive: by how much? https://www.youtube.com/watch?v=5E0LU85EnTI&t=1025s https://www.youtube.com/watch?v=5E0LU85EnTI&t=1025s
        fulafel 3y ago
        On a wimpy old desktop running lots of other stuff I got 75000 threads with that snippet (had to increase max_map_count tunable first with sysctl -w vm.max_map_count=500000, a knob well documented for bigger thread counts). Considering that in a real world use case with that much concurrency (such as the "100k threads frequently waiting for 100ms db queries" scenario), I'd be using a bigger machine and there'd also be some actual application context data and TCP connection state dwarfing the thread memory requirements in those blocked contexts, I'll still call virtual threads solving quite rare use cases, pending stronger evidence. I don't doubt virtual threads are efficient when microbenchmarked against OS threads, but there's only so much to be gained when optimizing a part of the system that wasn't a bottleneck to begin with. (Also I hope in the scenario we have a beefy DB that can exploit the concurrency available in this many pending concurrent queries per client, and isn't just putting them in ever queue! I guess it could, maybe it has 1000 read replica servers, each with 100 attached SSDs or 100 cores serving from in-memory data)
- vips7L 3y ago
  As pron always says.. that's a risk of staying on an LTS release. You're choosing to not receive updates.
  - geodel 3y ago
    It seems to me by design or by indifference Java is now tired of JVM ecosystem languages at large who claim innovation by surface level changes while relying heavily on JVM bedrock. Now Java is moving fast and they are taking JVM compiler, runtime, class metadata etc along where Java, the language is headed.
    - pjmlp 3y ago
      Not only Java, that is the fate of all guest languages, as they don't evolve alongside the platform, always require additional tooling and most language cultures than start making their own little islands of idiomatic libraries instead of directly using platform libraries. Since most platforms are leaky, one ends up needing to master two ecosystem in parallel, eventually it gets tiring and always cost extra in development. C++, Objective-C and Typescript get around this problem in UNIX and Web, as they are extensions of the platform languages, not something completely different.
      - geodel 3y ago
        Yep agree with all you said. Nowadays new job every year people at work are scaring me with their enthusiasm to replace decades old solid Java applications which are performant and working fine with Kotlin and what not.
        Alupis 3y ago
        I wouldn't advocate for replacing/re-writing old stuff - but you should try out Kotlin for backend work. It's amazing. Java programmers will have only a small learning curve before feeling productive, and can gradually get more comfortable with idomatic Kotlin as they progress through their journey. Kotlin, as a language, was very well through out, letting you use as much or as little of it as you are comfortable with.
- koito17 3y ago
  In fact, the officially supported Java releases for Clojure are all LTS releases. This is also reflected in their CI. At the moment, that means 1.8, 11, 17, and 21. I would also like to mention, for non-Clojure users: it took ages for Clojure to finally get off Java 1.5. The compiler was still emitting 1.5 compatible bytecode as late as 2018. Since the release of Clojure 1.10 (17 December 2018), the minimum Java version was bumped up to 1.8, in the sense that the compiler now emits 1.8-compatible bytecode. There is a strong emphasis on moving slowly and NOT breaking things in the Clojure community. Likewise, if you look at the most recent Clojure survey results, the vast majority of people were still on 1.8 and 11 releases.[1] [1] See Q24 in https://www.surveymonkey.com/stories/SM-_2BH3b49f_2FXEkUlrb_2BJSThxg_3D_3D/ https://www.surveymonkey.com/stories/SM-_2BH3b49f_2FXEkUlrb_...
  - tombert 3y ago
    I actually do feel like this is partly what's responsible for Clojure's relative success in industry. If you have a company with a ton of Java 5 code, and you want some of the sexy functional programming features, your options basically boil down to "upgrade everything to the newer Java 8 to get maps and optionals and whatnot" or "just write this new service in Clojure that will happily interop with our Java 5 code". I'm not making a judgement on which decision is better, but I can totally see the appeal, at least in the short term, for people choosing Clojure, especially if they think that Clojure's API will be largely unchanged in the Java 8 update. It doesn't hurt that Clojure is just broadly a really pleasant to work with, at least in my opinion.
    - Quekid5 3y ago
      > If you have a company with a ton of Java 5 code... You are in deep deep trouble already, just process-wise. If you're making money hand over fist you might be able to afford to do that, but choice of language is not even on the radar in terms of what needs fixing. Now, if you mean in terms of digging out of that hole... just upgrade to newer dependencies, newer JDK versions, etc. That's the way to better performance, better memory usage, better GC, better everything... and THEN think about the language you're using.
puredanger 3y ago
We are aware, and that is one of the options. :)
- puredanger 3y ago
  The other thing I didn't mention is that the change to remove biased locking a while back has also had an impact in some places that do almost always uncontended locking, so we're kind of considering that too.

theanonymousone 3y ago

Still waiting for VS Code to support Java 21. Java is not the language to use without an IDE.

jgon 3y ago
Brian Goetz just gave a talk about all the stuff that's been delivered up to Java 21 that announced they'd be rolling out a fully supported VS Code plugin for Java that should be better than the current language support. It should be out within the next week or two. Timestamp for the announcement: https://youtu.be/eXCx2hW_xNI?t=1955 https://youtu.be/eXCx2hW_xNI?t=1955
vips7L 3y ago
Things take time. If you want you can contribute resources to the vs code plugin and the eclipse language server.
avodonosov 3y ago
What in the java language makes you think it is less suitable to be used without an IDE than other languages?
- matsemann 3y ago
  It used to be lots of boilerplate. Like MyClassType myClassType = new MyClassType() And then it was nice to use an IDE to help write some of that. Or generate java beans etc. But with that said, I don't think it's much you need an IDE for, it's more that java enables IDEs to do much more than it can for certain other languages.
  - vips7L 3y ago
    You haven't had to declare the type since Java 10. var myThing = new MyThing();
    - riku_iki 3y ago
      still need it for class members, unlike in some other languages.

avodonosov 3y ago

Strange that synchnronization is used at all in lazy-seq The docs do not mention any synchronization: https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/lazy-seq https://clojure.github.io/clojure/clojure.core-api.html#cloj...

lcedp 3y ago
> a Seqable object that will invoke the body only the first time seq is called, and will cache the result and return it on all subsequent seq calls. To guarantee "only the first time", I suppose you need to have some kind of synchronization at some level.
- avodonosov 3y ago
  For multithreaded access yes. I meant that the docs do not mention thread safety.
  - daveliepmann 3y ago
    "All of the Clojure collections...are efficient and inherently thread-safe." https://clojure.org/reference/data_structures#Collections https://clojure.org/reference/data_structures#Collections "Seqs differ from iterators in that they are persistent and immutable, not stateful cursors into a collection. As such, they are useful for much more than foreach - functions can consume and produce seqs, they are thread safe, they can share structure etc." https://clojure.org/reference/sequences https://clojure.org/reference/sequences I think it's a fundamental enough property of clojure data structures that it would be redundant to mention it in every docstring.
    - avodonosov 3y ago
      Thanks, that's convincing. (why I was in doubt, is because while immutable functional collections that clojure embaces are naturally thread-safe, there is an imperative aspect in creation of a lazy sequence from user provided function, and I was not sure clojure has to provide synchronization guarantees here)