4 ms·

Let's Take a Look at JEP 483: Ahead-of-Time Class Loading and Linking

50 points by ingve 2y ago

robertlagrant 2y ago
> The AOT cache file has a size of 66 MB in this case. It is considered an implementation detail and as such is subject to change between Java versions. Does this mean this is mostly focused on containerised environments? I assume it means you couldn't upgrade the JVM on a server and know that this will keep working?
- gunnarmorling 2y ago
  Leyden's AOT support is not specifically geared towards containerized workloads, AFAICS. It's a general trend in the Java world though to bundle the JVM alongside applications, potentially creating a tailored JVM distribution with just a subset of all the available modules of the SDK. The model where one JVM is installed on a server and multiple applications are using it is becoming more and more rare. In that current model, a JVM upgrade would happen when distributing a new version of the application using it.
  - robertlagrant 2y ago
    I've been out of the Java world a while - you spotted exactly what I was thinking of, which was a global JVM deployment that everything used. Or at least an app container that lots of things used. Thanks for the update.
ignoramous 2y ago
> While start-up times don't matter that much for long running workloads, they can make a huge difference in cloud-native scenarios where applications are dynamically scaled out, spinning up new instances on demand as the load of incoming requests increases. "App Images" are what these are called in Android afaik. ART (the Android Run Time) has been capable of "AoT Class Loading & Linking" since 2016/7 (and ~pauseless GC since 2019). Seems like the desktop/server grade JVMs have a lot of ground to cover.
- pjmlp 2y ago
  On the contrary, lots of this work was already present in commercial products long before Android came to be, https://www.ptc.com/en/products/developer-tools/perc https://www.ptc.com/en/products/developer-tools/perc https://www.aicas.com/products-services/jamaicavm/ https://www.aicas.com/products-services/jamaicavm/ https://eclipse.dev/openj9/docs/aot/ https://eclipse.dev/openj9/docs/aot/ https://www.ibm.com/docs/en/sdk-java-technology/8?topic=reference-aot-compiler https://www.ibm.com/docs/en/sdk-java-technology/8?topic=refe... Now gone, https://en.wikipedia.org/wiki/Excelsior_JET https://en.wikipedia.org/wiki/Excelsior_JET What you are getting now is commercial work being offered as free beer, also why Excelsior JET is no longer relevant as company.
  - ignoramous 2y ago
    Thanks. > On the contrary, lots of this work was already present in commercial products I should have been clearer that I meant other OSS JVMs have catching up to do. > long before Android True. The evolution of Dalvik/ART remind me of HP Dynamo: https://archive.arstechnica.com/reviews/1q00/dynamo/dynamo-2.html https://archive.arstechnica.com/reviews/1q00/dynamo/dynamo-2... > What you are getting now is commercial work being offered as free beer ... AOSP is free as in beer.
    - pjmlp 2y ago
      > AOSP is free as in beer. True, however it isn't Java proper, and apparently Google has started to rewrite subsystems in Kotlin as well, when initially they said the OS layers would still be Java (language). Apparently the updates to Java 11 and 17 LTS subsets in ART, were motivated to keep up with Maven Central more than anything else.
- pron 2y ago
  Both the requirements and the workload challenges of server-side applications differ greatly from client applications. The JDK has had a pauseless GC (i.e. <1ms pauses) for 18 months now. The need for fast startup and warmup in server-side apps is newer than for client apps, and even with it, optimising for peak performance is still a higher priority than for client apps. You should expect a runtime that is used more on the server side to be "behind" when it comes to concerns that are more prevalent on the client and vice-versa, and that's exactly what you see.
cleverfoo 2y ago
I think the big problem here is conceptual. The JDK folks are looking at this akin to PGO when, IMHO, they should be looking at this as an AOT cache (yes, the flag names make this even more confusing). How do those two differ, you ask? With PGO you do a lot of deliberate work to profile your application under different conditions and feed that information back to the compiler to make better branch/inlining decisions. With a AOT cache, you do nothing up front, and the JVM should just dump a big cache to disk every time it exits just in case it gets stared again on the same host. In this case, training runs would just be a” run you did to create the cache". With that said, the big technical challenge right ow is that building the AOT cache is expensive hence performance impacting and cannot really be done alongside a live application - but that’s where I think the focus should be, making filling the aot cache something less intensive and automatic. Another aspect this strategy would help with is “what to do with these big AOT cache files”, if the AOT cache really starts caching every compiled method, it will become essentially another so file possibly of a size greater than the original JAR it started off with. Keeping this is in a docker image will double the size of the image slowing down deployments. Alternatively, with the aot cache concept, you just need to ensure there is some form of persistent disk cache across your hosts. The same logic also significantly helps CLIs, where I dont’ want to ship a 100MB CLI + Jlink bundle and have to add another 50MB of aot cache in it - what I do want is every time the client uses my CLI the JVM keeps improving the AOT cache.
- jagged-chisel 2y ago
  Is there any reason to think Java code can’t be statically linked, and then dead code eliminated (for that specific build of the app)? I’m not asking if the tooling currently exists, I’m curious if there’s something inherent in .class files that would prevent static linking.
  - pron 2y ago
    Native Image does exactly that already, but producing an AOT-compiled-and-linked native executable is not the goal, it's just a means to some goal. The real question is what is it that you want to optimise? Is it startup/warmup time? Is it the size of the binary? Peak performance? Developer productivity? Program functionality? Ops capabilities? AOT compilation certainly doesn't give you the best outcome for all of these concerns (regardless of the language).

50 points by ingve 2y ago

> The AOT cache file has a size of 66 MB in this case. It is considered an implementation detail and as such is subject to change between Java versions. Does this mean this is mostly focused on containerised environments? I assume it means you couldn't upgrade the JVM on a server and know that this will keep working?

gunnarmorling 2y ago
Leyden's AOT support is not specifically geared towards containerized workloads, AFAICS. It's a general trend in the Java world though to bundle the JVM alongside applications, potentially creating a tailored JVM distribution with just a subset of all the available modules of the SDK. The model where one JVM is installed on a server and multiple applications are using it is becoming more and more rare. In that current model, a JVM upgrade would happen when distributing a new version of the application using it.
- robertlagrant 2y ago
  I've been out of the Java world a while - you spotted exactly what I was thinking of, which was a global JVM deployment that everything used. Or at least an app container that lots of things used. Thanks for the update.

ignoramous 2y ago

> While start-up times don't matter that much for long running workloads, they can make a huge difference in cloud-native scenarios where applications are dynamically scaled out, spinning up new instances on demand as the load of incoming requests increases. "App Images" are what these are called in Android afaik. ART (the Android Run Time) has been capable of "AoT Class Loading & Linking" since 2016/7 (and ~pauseless GC since 2019). Seems like the desktop/server grade JVMs have a lot of ground to cover.

pjmlp 2y ago
On the contrary, lots of this work was already present in commercial products long before Android came to be, https://www.ptc.com/en/products/developer-tools/perc https://www.ptc.com/en/products/developer-tools/perc https://www.aicas.com/products-services/jamaicavm/ https://www.aicas.com/products-services/jamaicavm/ https://eclipse.dev/openj9/docs/aot/ https://eclipse.dev/openj9/docs/aot/ https://www.ibm.com/docs/en/sdk-java-technology/8?topic=reference-aot-compiler https://www.ibm.com/docs/en/sdk-java-technology/8?topic=refe... Now gone, https://en.wikipedia.org/wiki/Excelsior_JET https://en.wikipedia.org/wiki/Excelsior_JET What you are getting now is commercial work being offered as free beer, also why Excelsior JET is no longer relevant as company.
- ignoramous 2y ago
  Thanks. > On the contrary, lots of this work was already present in commercial products I should have been clearer that I meant other OSS JVMs have catching up to do. > long before Android True. The evolution of Dalvik/ART remind me of HP Dynamo: https://archive.arstechnica.com/reviews/1q00/dynamo/dynamo-2.html https://archive.arstechnica.com/reviews/1q00/dynamo/dynamo-2... > What you are getting now is commercial work being offered as free beer ... AOSP is free as in beer.
  - pjmlp 2y ago
    > AOSP is free as in beer. True, however it isn't Java proper, and apparently Google has started to rewrite subsystems in Kotlin as well, when initially they said the OS layers would still be Java (language). Apparently the updates to Java 11 and 17 LTS subsets in ART, were motivated to keep up with Maven Central more than anything else.
pron 2y ago
Both the requirements and the workload challenges of server-side applications differ greatly from client applications. The JDK has had a pauseless GC (i.e. <1ms pauses) for 18 months now. The need for fast startup and warmup in server-side apps is newer than for client apps, and even with it, optimising for peak performance is still a higher priority than for client apps. You should expect a runtime that is used more on the server side to be "behind" when it comes to concerns that are more prevalent on the client and vice-versa, and that's exactly what you see.

cleverfoo 2y ago

I think the big problem here is conceptual. The JDK folks are looking at this akin to PGO when, IMHO, they should be looking at this as an AOT cache (yes, the flag names make this even more confusing). How do those two differ, you ask? With PGO you do a lot of deliberate work to profile your application under different conditions and feed that information back to the compiler to make better branch/inlining decisions. With a AOT cache, you do nothing up front, and the JVM should just dump a big cache to disk every time it exits just in case it gets stared again on the same host. In this case, training runs would just be a” run you did to create the cache". With that said, the big technical challenge right ow is that building the AOT cache is expensive hence performance impacting and cannot really be done alongside a live application - but that’s where I think the focus should be, making filling the aot cache something less intensive and automatic. Another aspect this strategy would help with is “what to do with these big AOT cache files”, if the AOT cache really starts caching every compiled method, it will become essentially another so file possibly of a size greater than the original JAR it started off with. Keeping this is in a docker image will double the size of the image slowing down deployments. Alternatively, with the aot cache concept, you just need to ensure there is some form of persistent disk cache across your hosts. The same logic also significantly helps CLIs, where I dont’ want to ship a 100MB CLI + Jlink bundle and have to add another 50MB of aot cache in it - what I do want is every time the client uses my CLI the JVM keeps improving the AOT cache.

jagged-chisel 2y ago
Is there any reason to think Java code can’t be statically linked, and then dead code eliminated (for that specific build of the app)? I’m not asking if the tooling currently exists, I’m curious if there’s something inherent in .class files that would prevent static linking.
- pron 2y ago
  Native Image does exactly that already, but producing an AOT-compiled-and-linked native executable is not the goal, it's just a means to some goal. The real question is what is it that you want to optimise? Is it startup/warmup time? Is it the size of the binary? Peak performance? Developer productivity? Program functionality? Ops capabilities? AOT compilation certainly doesn't give you the best outcome for all of these concerns (regardless of the language).