4 ms·

JEP 483: Ahead-of-Time Class Loading and Linking

148 points by ptx 2y ago

petesoper 2y ago
Sweet!
foolfoolz 2y ago
i’m curious if any of this was inspired from aws lambda snapstart
- layer8 2y ago
  Maybe read the History section.
layer8 2y ago
> [example hello world] program runs in 0.031 seconds on JDK 23. After doing the small amount of additional work required to create an AOT cache it runs in in 0.018 seconds on JDK NN — an improvement of 42%. The AOT cache occupies 11.4 megabytes. That’s not immediately convincing that it will be worth it. It is a start I guess.
- dgfitz 2y ago
  How so? RAM is almost free if you’re not on embedded, and embedded could run Java sure, but it isn’t common.
  - pdpi 2y ago
    That’s not an in-memory cache either. AIUI it’s storing those artefacts to disk
    - lmz 2y ago
      Container sizes may be affected though.
      - bobnamob 2y ago
        So you're now weighing the increased container pull time (due to size) vs the class load time you're saving through the cache. It's nice to at least have the option of making that tradeoff (And I suspect for plenty of applications, the class cache will be worth more time than (an also probably cached) image pull)
      - pdpi 2y ago
        If you’re deploying Java applications, container size isn’t exactly your first priority anyhow, and this is O(n) additional space. If image size is a concern, I imagine a native binary using GraalVM would’ve been a better way out anyhow, and you’ll bypass this cache entirely.
  - imtringued 2y ago
    RAM might be inexpensive, but this hasn't stopped cloud providers from being stingy with RAM and price gouging. At current RAM prices you'd expect the smallest instances to have 2GB, yet they still charge $4/month for 512MB, which isn't enough to run the average JVM web server.
    - zokier 2y ago
      That is pretty ridiculous complaint. Your problem is that they allow configuring instances smaller than your arbitrary baseline? Especially as AWS allows you to pick 2/4/8 GB per vCPU for general purpose instances. And the smallest of these (c7g.medium) is 2GB/1vCPU. The .5 GB t4g.nano has actually more generous ratio because it also has only .1 vCPU, putting it at 5GB/vCPU. I'd assume they are very aware of demand levels for different types and would be adjusting the configurations if needed.
o11c 2y ago
The concern that jumps out at me is: what about flags that affect code generation? Some are tied to the subarch (e.g. "does this amd64 have avx2?" - relevant if the cache is backed up and restored to a slightly different machine, or sometimes even if it reboots with a different kernel config), others to java's own flags (does compressed pointers affect codegen? disabling intrinsics?).
- lxgr 2y ago
  I don’t see any mention that code is actually going to be stored in a JITted form, so possibly it’s just architecture-independent loading and linking data being cached?
  - MBCook 2y ago
    My impression from reading this was it was about knowing which classes reference which other classes when and which jars everything is in. So I think you’re right. So a bit more linker style optimization than compiler related caching stuff.
    - brabel 2y ago
      The JEP explains what this does: "The AOT cache builds upon CDS by not only reading and parsing class files ahead-of-time but also loading and linking them." While CDS (which has been available for years now) only caches a parsed form of the class files that got loaded by the application, the AOT cache will also "load and link" the classes. The ClassLoader.load method docs explain what loading means: https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/ClassLoader.html#loadClass(java.lang.String,boolean) https://docs.oracle.com/en/java/javase/21/docs/api/java.base... 1. find the class (usually by looking at the file-index of the jar, which is just a zip archive, but ClassLoaders can implement this in many ways). 2. link the class, which is done by the resolveClass method: https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/ClassLoader.html#resolveClass(java.lang.Class) https://docs.oracle.com/en/java/javase/21/docs/api/java.base... and explained in the Java Language Specification: https://docs.oracle.com/javase/specs/jls/se21/html/jls-12.html#jls-12.3 https://docs.oracle.com/javase/specs/jls/se21/html/jls-12.ht... "Three different activities are involved in linking: verification, preparation, and resolution of symbolic references." Hence, I assume the AOT cache will somehow keep even symbolic references between classes, which is quite interesting.
fulafel 2y ago
What does this mean for Clojure? At least loading the Clojure runtime should benefit, but what about app code loading.
- funcDropShadow 2y ago
  It should benefit, if namespaces are AOT-compiled by Clojure.
- diggan 2y ago
  I feel like for the Clojure applications where you need it to start really fast, like tiny CLI utilities that don't do a lot of work, the improvements would be so marginal to not matter much. The example they use in the JEP seems to have gone from a ~4 second startup to ~2 seconds, which for a tiny CLI, still would make it seem pretty slow. You're better off trying to use Babashka, ClojureScript or any of the other solutions that give a fast startup. And for the bigger applications (like web services and alike), you don't really care that it takes 5 seconds or 10 seconds to start it, you only restart the server during deployment anyways, so why would startup time matter so much?
  - dtech 2y ago
    The 4 second application is a web server. They also give a basic example starting in 0.031s, fine for a CLI. One of the use cases for startup time is AWS lambda and similar.
    - bobnamob 2y ago
      Prebuilding a cache through a training run will be difficult between lambda invocations though and snapstart[1] already "solves" a lot of the issues a class cache might address. [1] https://docs.aws.amazon.com/lambda/latest/dg/snapstart.html https://docs.aws.amazon.com/lambda/latest/dg/snapstart.html Of course, I wouldn't be surprised if the boffins at lambda add some integration between snapstart and class caching once their leadership can get it funded
      - 2y ago
s6af7ygt 2y ago
I'm a dunce
- dtech 2y ago
  Read the article, this doesn't reduce JIT capabilities at all.
VonNaturAustre 2y ago
Nice, any time optimization is welcome :)
carimura 2y ago
For reference this is part of Project Leyden [1][2] which was introduced in 2022 with the first ideas showcased at JVMLS 2023 [3]. [1] https://openjdk.org/projects/leyden/ https://openjdk.org/projects/leyden/ [2] https://inside.java/tag/leyden https://inside.java/tag/leyden [3] https://www.youtube.com/watch?v=lnth19Kf-x0 https://www.youtube.com/watch?v=lnth19Kf-x0
pjmlp 2y ago
Note that OpenJ9 and Azul already do similar optimizations. And Android while not being a Java/JVM proper, more of a cousin, also has similar JIT cache concept as intermediate step before doing AOT compilation of selected code. Naturally also welcomed on the OpenJDK distributions.

148 points by ptx 2y ago

petesoper 2y ago

Sweet!

foolfoolz 2y ago

i’m curious if any of this was inspired from aws lambda snapstart

layer8 2y ago
Maybe read the History section.

layer8 2y ago

> [example hello world] program runs in 0.031 seconds on JDK 23. After doing the small amount of additional work required to create an AOT cache it runs in in 0.018 seconds on JDK NN — an improvement of 42%. The AOT cache occupies 11.4 megabytes. That’s not immediately convincing that it will be worth it. It is a start I guess.

dgfitz 2y ago
How so? RAM is almost free if you’re not on embedded, and embedded could run Java sure, but it isn’t common.
- pdpi 2y ago
  That’s not an in-memory cache either. AIUI it’s storing those artefacts to disk
  - lmz 2y ago
    Container sizes may be affected though.
    - bobnamob 2y ago
      So you're now weighing the increased container pull time (due to size) vs the class load time you're saving through the cache. It's nice to at least have the option of making that tradeoff (And I suspect for plenty of applications, the class cache will be worth more time than (an also probably cached) image pull)
    - pdpi 2y ago
      If you’re deploying Java applications, container size isn’t exactly your first priority anyhow, and this is O(n) additional space. If image size is a concern, I imagine a native binary using GraalVM would’ve been a better way out anyhow, and you’ll bypass this cache entirely.
- imtringued 2y ago
  RAM might be inexpensive, but this hasn't stopped cloud providers from being stingy with RAM and price gouging. At current RAM prices you'd expect the smallest instances to have 2GB, yet they still charge $4/month for 512MB, which isn't enough to run the average JVM web server.
  - zokier 2y ago
    That is pretty ridiculous complaint. Your problem is that they allow configuring instances smaller than your arbitrary baseline? Especially as AWS allows you to pick 2/4/8 GB per vCPU for general purpose instances. And the smallest of these (c7g.medium) is 2GB/1vCPU. The .5 GB t4g.nano has actually more generous ratio because it also has only .1 vCPU, putting it at 5GB/vCPU. I'd assume they are very aware of demand levels for different types and would be adjusting the configurations if needed.

o11c 2y ago

The concern that jumps out at me is: what about flags that affect code generation? Some are tied to the subarch (e.g. "does this amd64 have avx2?" - relevant if the cache is backed up and restored to a slightly different machine, or sometimes even if it reboots with a different kernel config), others to java's own flags (does compressed pointers affect codegen? disabling intrinsics?).

lxgr 2y ago
I don’t see any mention that code is actually going to be stored in a JITted form, so possibly it’s just architecture-independent loading and linking data being cached?
- MBCook 2y ago
  My impression from reading this was it was about knowing which classes reference which other classes when and which jars everything is in. So I think you’re right. So a bit more linker style optimization than compiler related caching stuff.
  - brabel 2y ago
    The JEP explains what this does: "The AOT cache builds upon CDS by not only reading and parsing class files ahead-of-time but also loading and linking them." While CDS (which has been available for years now) only caches a parsed form of the class files that got loaded by the application, the AOT cache will also "load and link" the classes. The ClassLoader.load method docs explain what loading means: https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/ClassLoader.html#loadClass(java.lang.String,boolean) https://docs.oracle.com/en/java/javase/21/docs/api/java.base... 1. find the class (usually by looking at the file-index of the jar, which is just a zip archive, but ClassLoaders can implement this in many ways). 2. link the class, which is done by the resolveClass method: https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/ClassLoader.html#resolveClass(java.lang.Class) https://docs.oracle.com/en/java/javase/21/docs/api/java.base... and explained in the Java Language Specification: https://docs.oracle.com/javase/specs/jls/se21/html/jls-12.html#jls-12.3 https://docs.oracle.com/javase/specs/jls/se21/html/jls-12.ht... "Three different activities are involved in linking: verification, preparation, and resolution of symbolic references." Hence, I assume the AOT cache will somehow keep even symbolic references between classes, which is quite interesting.

fulafel 2y ago

What does this mean for Clojure? At least loading the Clojure runtime should benefit, but what about app code loading.

funcDropShadow 2y ago
It should benefit, if namespaces are AOT-compiled by Clojure.
diggan 2y ago
I feel like for the Clojure applications where you need it to start really fast, like tiny CLI utilities that don't do a lot of work, the improvements would be so marginal to not matter much. The example they use in the JEP seems to have gone from a ~4 second startup to ~2 seconds, which for a tiny CLI, still would make it seem pretty slow. You're better off trying to use Babashka, ClojureScript or any of the other solutions that give a fast startup. And for the bigger applications (like web services and alike), you don't really care that it takes 5 seconds or 10 seconds to start it, you only restart the server during deployment anyways, so why would startup time matter so much?
- dtech 2y ago
  The 4 second application is a web server. They also give a basic example starting in 0.031s, fine for a CLI. One of the use cases for startup time is AWS lambda and similar.
  - bobnamob 2y ago
    Prebuilding a cache through a training run will be difficult between lambda invocations though and snapstart[1] already "solves" a lot of the issues a class cache might address. [1] https://docs.aws.amazon.com/lambda/latest/dg/snapstart.html https://docs.aws.amazon.com/lambda/latest/dg/snapstart.html Of course, I wouldn't be surprised if the boffins at lambda add some integration between snapstart and class caching once their leadership can get it funded
    - 2y ago

s6af7ygt 2y ago

I'm a dunce

dtech 2y ago
Read the article, this doesn't reduce JIT capabilities at all.

VonNaturAustre 2y ago

Nice, any time optimization is welcome :)

carimura 2y ago

For reference this is part of Project Leyden [1][2] which was introduced in 2022 with the first ideas showcased at JVMLS 2023 [3]. [1] https://openjdk.org/projects/leyden/ https://openjdk.org/projects/leyden/ [2] https://inside.java/tag/leyden https://inside.java/tag/leyden [3] https://www.youtube.com/watch?v=lnth19Kf-x0 https://www.youtube.com/watch?v=lnth19Kf-x0

pjmlp 2y ago

Note that OpenJ9 and Azul already do similar optimizations. And Android while not being a Java/JVM proper, more of a cousin, also has similar JIT cache concept as intermediate step before doing AOT compilation of selected code. Naturally also welcomed on the OpenJDK distributions.