8 ms·

Ahead-of-Time Compilation

128 points by hittaruki 10y ago

taspeotis 10y ago
I can't see anywhere in the linked issue that indicates AOT compilation is coming to Java 9, or even coming at all. The issue demonstrates nothing more than an intent to bring it to OpenJDK, and the issue seems to be very nascent? It was only created a fortnight ago. Lest the title is changed: AOT compilation is coming to Java 9 (java.net) 18 points by hittaruki 37 minutes ago
- chc 10y ago
  The person who created this ticket is an Oracle employee, not some random Joe, so it seems like a reasonable guess that it's something Oracle is planning.
  - pjmlp 10y ago
    They are planning it, and there were already several talks, but the roadmap was Java 10 or later.
  - mike_hearn 10y ago
    There was a talk about it last year: https://www.youtube.com/watch?v=Xybzyv8qbOc The project seems to have gone slower than I expected, perhaps because Chris Thalinger moved to Twitter.
- dantiberian 10y ago
  > The extra step of recompiling code at Tier 3 is necessary since the overhead of full profiling is too high to be used for all methods, especially for a module such as java.base. For user applications it might make sense to allow AOT compilations with Tier 3-equivalent profiling, but this will not be supported for JDK 9. This implies it will be in Java 9 (in a limited fashion).
johnydepp 10y ago
That's great! Won't it make the compiled executable platform specific?
- rcaught 10y ago
  Write once, run anywhere... you've compiled it to.
- jcdavis 10y ago
  the generated .so file will be platform specific, but it seems like the class files will still be needed as usual
- copperx 10y ago
  I believe that's the whole point. From what I could gather, this is the process one would follow to get native code: .java -> javac -> .class (still cross-platform bytecode) -> jaotc -> .so native code
- Sanddancer 10y ago
  There are other JVMs that already do AOT compilation. This just brings an AOT compilation option to Java. It also looks like the contained file will include both the compiled version and the normal bytecode, so that the code can be recompiled if/when necessary.
- mike_hearn 10y ago
  You're intended to use it like this: either you distribute JARs and the recipient triggers the AOT compilation if they want it, or you distribute a "jlinked" JRE image that's inherently OS specific because it includes a bundled JVM. It's also possible that a future Java module format will allow the AOT compiled code images to come along for the ride next to the classfiles.
avbor 10y ago
I'm not familiar enough with compilers, but why would an ahead of time compiler perform worse than a just in time compiler in a static language? I think I'd understand if it was a dynamic language, because you can't know the types for sure until you start running the program, but are similar issues present for Java?
- Scarbutt 10y ago
  A JIT can use information from runtime for optimizations, you pay a cost in startup time. Being worse or better depends on the workloads and implementations.
  - CalChris 10y ago
    AOT can as well. PGO.
    - alblue 10y ago
      There are a set of optimisations that you can only do at runtime that you can't do with AOT. Anything that depends on data is something that you can't reliably do, such as eliding null checks if you can prove this cannot happen based on the data passed in. There are also cases where you can have multiple subclasses of a type such as a normal and a debug subclass, or multiple drivers for different back ends such as MongoDB or Cassandra, only one of which is used at runtime but you cannot know ahead of time which is selected (for example, it's based on an environment variable or system property). The point is that while AOT can do a set of optimisations, including whole module analysis, there are a set which are only available at runtime.
    - vidarh 10y ago
      Yes, but that locks you in to optimising for whatever you covered with your profiling. If the character of your data changes, it'll take a recompile to change how the application performs, while a JIT can potentially choose to deoptimise/reoptimise. I'm all for "as static as possible" toolchains, but there are optimization opportunities you simply won't have with AOT, PGO or not. E.g. consider something trivial: A program doing certain image operations that depends on dimensions passed in on the command line. A JIT could optimise the inner loops for the actual operations. To get the same with AOT even with PGO would be totally unable to deal with it without causing a massive explosion in code size.
Cyph0n 10y ago
Assuming this comes in Java 9, and compilation of code other `java.base` is possible, will this make Java a more solid competitor to Go? I guess it partly depends on how much they optimize the compiled binary size. Go does a really good job at static compilation, so it will be tough to compete.
- paukiatwee 10y ago
  Java AOT is compile JVM bytecode to native code during startup of JVM, which is different from Go's compile source to native and distribute platform specific binaries. So in this case, Java binary size remain same as before, which is .jar or .war binaries. For Go, .go -> native For Java, .java -> .class -> package .jar -> AOT native For Go part I might be wrong, not working on Go professionally.
  - sandGorgon 10y ago
    that is an incorrect assumption. AOT step will include deadcode elimination. In that way - the Java way is a more sophisticated way combining platform independent bytecode and platform-specific machine code.
  - Scarbutt 10y ago
    My understanding is that JVM bytecode is compiled to native before startup of the JVM, that is why it is called AOT ;)
  - bitmapbrother 10y ago
    No it's not. Compilers such as the one in IBM's J9 have a switch setting to use AOT.
  - mike_hearn 10y ago
    No, not during startup, AOT can happen at any time chosen by the developer or user: there's a command line tool that triggers it and I believe the plan is to integrate it with the "jlink" tool that produces standalone, app specific JRE images. So you can produce a native installer for each platform.
- 10y ago
haberman 10y ago
How does this interact with classloading? My general impression is that the design of classloaders is pretty actively hostile to making JVM startup fast.
- qznc 10y ago
  Yes it is. On the other hand, gcj still did it years ago. I guess some features like dynamic class loading are just not supported.
- the8472 10y ago
  AOT only applies to clases that have been AOT-compiled and are not transformed at runtime. Everything else will either still be need JITing or could potentially throw errors if pure AOT is desired. AOT and JIT are not mutually exclusive. From the proposal itself: > AOT libraries can be compiled in two modes: > Non-tiered AOT compiled code behaves similarly to statically compiled C++ code in that no profiling information is collected and no JIT recompilations will happen. > Tiered AOT compiled code does collect profiling information. The profiling done is the same as the simple profiling done by C1 methods compiled at Tier 2. If AOT methods hit the AOT invocation thresholds these methods are being recompiled by C1 at Tier 3 first in order to gather full profiling information. This is required for C2 JIT recompilations to be able to produce optimal code and reach peak application performance.
saynsedit 10y ago
not sure why this isn't a transparent feature implemented via caching.
- jcdavis 10y ago
  Hotspot's JIT compiled code tends to be pretty specialized based on runtime profiling information, which may not necessarily be similar between different runs even the class itself hasn't changed, or (in an extreme case) even if none of the code has. Some other JVMs (at least Azul's Zing) try to solve this by cache profiling information to speed up code generation.
  - alblue 10y ago
    I believe the ReadyNow technology used by Zing records what methods are compiled at which level, then trigger a compilation of those methods at start up. So you effectively use profiling information from the previous run to inform the next run of what the final target state is, allowing the warm up times to be dramatically reduced.
- mike_hearn 10y ago
  It's explained in the talk "Java goes AOT": https://www.youtube.com/watch?v=Xybzyv8qbOc https://www.youtube.com/watch?v=Xybzyv8qbOc Basically they thought it'd de-opt too much. I'm not totally sure it's the case but they'd be the experts on that.
KMag 10y ago
This is great news, though it initially only supports Linux x86-64 and is decades late for Java desktop apps (and not having non-blocking I/O until Java 1.4 was shameful for a language explicitly targeted and a pervasively networked ecosystem.) In their "tiered mode", they put sampling instrumentation into the native code, and if they detect a hotspot, regenerate fully instrumented native code from bytecode using the C1 (fast) JIT, which then allows the C2 JIT to do its full optimizations on the code as if AoT were not involved. Since the invention of tracing JITs, I've often wondered why languages don't package together a compact serialized SSA form such as LLVM bitcode or SafeTSA along with storing functions as lists of pointers to space-optimized compilations of extended basic blocks (strait-line code), similar to how some Forth compilers generate threaded code. A threaded code dispatcher over these strait-line segments of native code would have minimal overhead, and when a simple SIGPROF lightweight sampler detected a hotspot, a tracing version of the dispatcher could collect a trace, and then generate native code from the visited traces using the stored SSA for the basic blocks. In this way, they'd have a light-weight tracing JIT for re-optimizing native code.
- nwmcsween 10y ago
  This could be taken even further, if the IR can hold about effects and purity, etc you could potentially optimize across libraries and binaries.
- pjmlp 10y ago
  > decades late for Java desktop apps Commercial JDKs always offered AOT compilation, the problem is that people nowadays apparently don't buy compilers anymore unless forced to do so (e.g. embedded, consoles...).
  - deleted 10y ago
    [deleted]
  - KMag 10y ago
    There has also been gcj for a long time, but default toolchains matter a lot.
BuckRogers 10y ago
Why was Java ever JIT'd rather than natively compiled anyway? I hate to stick my neck out and even ask this but I never understood why you'd want to JIT or interpret when you can just natively compile to a binary. It seems like Go has gone "back" to the future on this one and in general their toolchain approach to me looked like the way. I always got the sense the world is waiting for a statically typed Python that compiles to native code with Go's CPU performance. I suppose Nim might fit that bill but a shame it doesn't have compatibility with Python's or even the extent of a language like Go's libraries. And if possible, an imperative language that interfaces with OTP. And that said, I can see why Erlang/Elixir wouldn't make as much sense or even work with native code AOT compilation due to it's feature set (thinking stuff like hot code reloading). But I've never grasped why Java or Python were better off with JIT or interpreters than AOT comp. Seems like a type system such as Go's is simple enough and allows for good gains in both CPU performance and memory usage. Add in the fact you don't need to install anything and less to think about in deploying and it seems to be a no brainer. Please feel free to fill me in on this or where I went wrong..
- jcheng 10y ago
  For several of Java's early use cases, being able to deploy a single file that could be run on any Java-supported platform was very important. https://en.wikipedia.org/wiki/Write_once,_run_anywhere https://en.wikipedia.org/wiki/Write_once,_run_anywhere
  - qznc 10y ago
    Also, a small file and stack based bytecode is often the most compact representation.
    - CalChris 10y ago
      This is true and small memory sizes and limited bandwidth were design assumptions valid in 1992.
- pvg 10y ago
  Deployment (and fever dreams of 'mobile code'). It's also worth remembering that Java was designed and implemented at a time when the landscape was significantly less x86-centric and Sun was one of the companies on the not-x86 side.
_ZeD_ 10y ago
it's a gcj comeback?
singularity2001 10y ago
resurrect me when it's there https://github.com/search?p=3&q=jaotc&type=Code https://github.com/search?p=3&q=jaotc&type=Code
alblue 10y ago
Slightly off topic but if you are interested in how HotSpot compiles to native code I gave a presentation at JavaOne: http://alblue.bandlem.com/2016/09/javaone-hotspot.html http://alblue.bandlem.com/2016/09/javaone-hotspot.html The presentation wasn't recorded but there is a video recorded from a DocklandsLJC event which is on InfoQ: https://www.infoq.com/presentations/hotspot-memory-data-structures https://www.infoq.com/presentations/hotspot-memory-data-stru...
my123 10y ago
IKVM, the Java VM for .NET converted Java bytecode to .NET since a while, and crossgen/ngen can be used for .NET AOT(Mono also has AOT)
jgalt212 10y ago
> Infrequently-used Java methods might never be compiled at all, potentially incurring a performance penalty due to repeated interpreted invocations. That sort of makes no sense. How can you incur a real performance hit if the uncompiled method is rarely called?
- smitherfield 10y ago
  I would assume if the method is rarely called, but very complex or time-consuming when it is called. Of course, I'm not an expert on JVMs, so I wouldn't know whether their analysis is synchronous or asynchronous or a mix of both.
premium-concern 10y ago
Is there anything this adds over Scala-native, which seems to be much further ahead already?
- edko 10y ago
  I think this could be a great complement to Scala-native. Right now, the project contributors have to spend effort translating the essential Java libraries that would allow Scala-native to be successful. This could really ease that job for them. It could potentially make all the Java code ever written available to Scala-native. The other thing it adds is the backing of a giant, like Oracle, which can bring stability and peace of mind to some people, when deciding whether to adopt the technology or not.
  - premium-concern 10y ago
    I think this assumes that Oracle - can ship something in time - and that it will be generally available for developers (looking at how hard Oracle pushes their Java department to invent commercial features they can sell, I'm not sure about that) Looking at it, I assume that this will go the way of GWT ... not starting from "how can we make Java a good citizen in this new ecosystem?", but "here we have 100% of Java, the JDK and the JVM ... how can we compile this with full fidelity into X?".

128 points by hittaruki 10y ago

I can't see anywhere in the linked issue that indicates AOT compilation is coming to Java 9, or even coming at all. The issue demonstrates nothing more than an intent to bring it to OpenJDK, and the issue seems to be very nascent? It was only created a fortnight ago. Lest the title is changed: AOT compilation is coming to Java 9 (java.net) 18 points by hittaruki 37 minutes ago

chc 10y ago
The person who created this ticket is an Oracle employee, not some random Joe, so it seems like a reasonable guess that it's something Oracle is planning.
- pjmlp 10y ago
  They are planning it, and there were already several talks, but the roadmap was Java 10 or later.
- mike_hearn 10y ago
  There was a talk about it last year: https://www.youtube.com/watch?v=Xybzyv8qbOc The project seems to have gone slower than I expected, perhaps because Chris Thalinger moved to Twitter.
dantiberian 10y ago
> The extra step of recompiling code at Tier 3 is necessary since the overhead of full profiling is too high to be used for all methods, especially for a module such as java.base. For user applications it might make sense to allow AOT compilations with Tier 3-equivalent profiling, but this will not be supported for JDK 9. This implies it will be in Java 9 (in a limited fashion).

johnydepp 10y ago

That's great! Won't it make the compiled executable platform specific?

rcaught 10y ago
Write once, run anywhere... you've compiled it to.
jcdavis 10y ago
the generated .so file will be platform specific, but it seems like the class files will still be needed as usual
copperx 10y ago
I believe that's the whole point. From what I could gather, this is the process one would follow to get native code: .java -> javac -> .class (still cross-platform bytecode) -> jaotc -> .so native code
Sanddancer 10y ago
There are other JVMs that already do AOT compilation. This just brings an AOT compilation option to Java. It also looks like the contained file will include both the compiled version and the normal bytecode, so that the code can be recompiled if/when necessary.
mike_hearn 10y ago
You're intended to use it like this: either you distribute JARs and the recipient triggers the AOT compilation if they want it, or you distribute a "jlinked" JRE image that's inherently OS specific because it includes a bundled JVM. It's also possible that a future Java module format will allow the AOT compiled code images to come along for the ride next to the classfiles.

avbor 10y ago

I'm not familiar enough with compilers, but why would an ahead of time compiler perform worse than a just in time compiler in a static language? I think I'd understand if it was a dynamic language, because you can't know the types for sure until you start running the program, but are similar issues present for Java?

Scarbutt 10y ago
A JIT can use information from runtime for optimizations, you pay a cost in startup time. Being worse or better depends on the workloads and implementations.
- CalChris 10y ago
  AOT can as well. PGO.
  - alblue 10y ago
    There are a set of optimisations that you can only do at runtime that you can't do with AOT. Anything that depends on data is something that you can't reliably do, such as eliding null checks if you can prove this cannot happen based on the data passed in. There are also cases where you can have multiple subclasses of a type such as a normal and a debug subclass, or multiple drivers for different back ends such as MongoDB or Cassandra, only one of which is used at runtime but you cannot know ahead of time which is selected (for example, it's based on an environment variable or system property). The point is that while AOT can do a set of optimisations, including whole module analysis, there are a set which are only available at runtime.
  - vidarh 10y ago
    Yes, but that locks you in to optimising for whatever you covered with your profiling. If the character of your data changes, it'll take a recompile to change how the application performs, while a JIT can potentially choose to deoptimise/reoptimise. I'm all for "as static as possible" toolchains, but there are optimization opportunities you simply won't have with AOT, PGO or not. E.g. consider something trivial: A program doing certain image operations that depends on dimensions passed in on the command line. A JIT could optimise the inner loops for the actual operations. To get the same with AOT even with PGO would be totally unable to deal with it without causing a massive explosion in code size.

Cyph0n 10y ago

Assuming this comes in Java 9, and compilation of code other `java.base` is possible, will this make Java a more solid competitor to Go? I guess it partly depends on how much they optimize the compiled binary size. Go does a really good job at static compilation, so it will be tough to compete.

paukiatwee 10y ago
Java AOT is compile JVM bytecode to native code during startup of JVM, which is different from Go's compile source to native and distribute platform specific binaries. So in this case, Java binary size remain same as before, which is .jar or .war binaries. For Go, .go -> native For Java, .java -> .class -> package .jar -> AOT native For Go part I might be wrong, not working on Go professionally.
- sandGorgon 10y ago
  that is an incorrect assumption. AOT step will include deadcode elimination. In that way - the Java way is a more sophisticated way combining platform independent bytecode and platform-specific machine code.
- Scarbutt 10y ago
  My understanding is that JVM bytecode is compiled to native before startup of the JVM, that is why it is called AOT ;)
- bitmapbrother 10y ago
  No it's not. Compilers such as the one in IBM's J9 have a switch setting to use AOT.
- mike_hearn 10y ago
  No, not during startup, AOT can happen at any time chosen by the developer or user: there's a command line tool that triggers it and I believe the plan is to integrate it with the "jlink" tool that produces standalone, app specific JRE images. So you can produce a native installer for each platform.
10y ago

haberman 10y ago

How does this interact with classloading? My general impression is that the design of classloaders is pretty actively hostile to making JVM startup fast.

qznc 10y ago
Yes it is. On the other hand, gcj still did it years ago. I guess some features like dynamic class loading are just not supported.
the8472 10y ago
AOT only applies to clases that have been AOT-compiled and are not transformed at runtime. Everything else will either still be need JITing or could potentially throw errors if pure AOT is desired. AOT and JIT are not mutually exclusive. From the proposal itself: > AOT libraries can be compiled in two modes: > Non-tiered AOT compiled code behaves similarly to statically compiled C++ code in that no profiling information is collected and no JIT recompilations will happen. > Tiered AOT compiled code does collect profiling information. The profiling done is the same as the simple profiling done by C1 methods compiled at Tier 2. If AOT methods hit the AOT invocation thresholds these methods are being recompiled by C1 at Tier 3 first in order to gather full profiling information. This is required for C2 JIT recompilations to be able to produce optimal code and reach peak application performance.

saynsedit 10y ago

not sure why this isn't a transparent feature implemented via caching.

jcdavis 10y ago
Hotspot's JIT compiled code tends to be pretty specialized based on runtime profiling information, which may not necessarily be similar between different runs even the class itself hasn't changed, or (in an extreme case) even if none of the code has. Some other JVMs (at least Azul's Zing) try to solve this by cache profiling information to speed up code generation.
- alblue 10y ago
  I believe the ReadyNow technology used by Zing records what methods are compiled at which level, then trigger a compilation of those methods at start up. So you effectively use profiling information from the previous run to inform the next run of what the final target state is, allowing the warm up times to be dramatically reduced.
mike_hearn 10y ago
It's explained in the talk "Java goes AOT": https://www.youtube.com/watch?v=Xybzyv8qbOc https://www.youtube.com/watch?v=Xybzyv8qbOc Basically they thought it'd de-opt too much. I'm not totally sure it's the case but they'd be the experts on that.

KMag 10y ago

This is great news, though it initially only supports Linux x86-64 and is decades late for Java desktop apps (and not having non-blocking I/O until Java 1.4 was shameful for a language explicitly targeted and a pervasively networked ecosystem.) In their "tiered mode", they put sampling instrumentation into the native code, and if they detect a hotspot, regenerate fully instrumented native code from bytecode using the C1 (fast) JIT, which then allows the C2 JIT to do its full optimizations on the code as if AoT were not involved. Since the invention of tracing JITs, I've often wondered why languages don't package together a compact serialized SSA form such as LLVM bitcode or SafeTSA along with storing functions as lists of pointers to space-optimized compilations of extended basic blocks (strait-line code), similar to how some Forth compilers generate threaded code. A threaded code dispatcher over these strait-line segments of native code would have minimal overhead, and when a simple SIGPROF lightweight sampler detected a hotspot, a tracing version of the dispatcher could collect a trace, and then generate native code from the visited traces using the stored SSA for the basic blocks. In this way, they'd have a light-weight tracing JIT for re-optimizing native code.

nwmcsween 10y ago
This could be taken even further, if the IR can hold about effects and purity, etc you could potentially optimize across libraries and binaries.
pjmlp 10y ago
> decades late for Java desktop apps Commercial JDKs always offered AOT compilation, the problem is that people nowadays apparently don't buy compilers anymore unless forced to do so (e.g. embedded, consoles...).
- deleted 10y ago
  [deleted]
- KMag 10y ago
  There has also been gcj for a long time, but default toolchains matter a lot.

BuckRogers 10y ago

Why was Java ever JIT'd rather than natively compiled anyway? I hate to stick my neck out and even ask this but I never understood why you'd want to JIT or interpret when you can just natively compile to a binary. It seems like Go has gone "back" to the future on this one and in general their toolchain approach to me looked like the way. I always got the sense the world is waiting for a statically typed Python that compiles to native code with Go's CPU performance. I suppose Nim might fit that bill but a shame it doesn't have compatibility with Python's or even the extent of a language like Go's libraries. And if possible, an imperative language that interfaces with OTP. And that said, I can see why Erlang/Elixir wouldn't make as much sense or even work with native code AOT compilation due to it's feature set (thinking stuff like hot code reloading). But I've never grasped why Java or Python were better off with JIT or interpreters than AOT comp. Seems like a type system such as Go's is simple enough and allows for good gains in both CPU performance and memory usage. Add in the fact you don't need to install anything and less to think about in deploying and it seems to be a no brainer. Please feel free to fill me in on this or where I went wrong..

jcheng 10y ago
For several of Java's early use cases, being able to deploy a single file that could be run on any Java-supported platform was very important. https://en.wikipedia.org/wiki/Write_once,_run_anywhere https://en.wikipedia.org/wiki/Write_once,_run_anywhere
- qznc 10y ago
  Also, a small file and stack based bytecode is often the most compact representation.
  - CalChris 10y ago
    This is true and small memory sizes and limited bandwidth were design assumptions valid in 1992.
pvg 10y ago
Deployment (and fever dreams of 'mobile code'). It's also worth remembering that Java was designed and implemented at a time when the landscape was significantly less x86-centric and Sun was one of the companies on the not-x86 side.

_ZeD_ 10y ago

it's a gcj comeback?

singularity2001 10y ago

resurrect me when it's there https://github.com/search?p=3&q=jaotc&type=Code https://github.com/search?p=3&q=jaotc&type=Code

alblue 10y ago

Slightly off topic but if you are interested in how HotSpot compiles to native code I gave a presentation at JavaOne: http://alblue.bandlem.com/2016/09/javaone-hotspot.html http://alblue.bandlem.com/2016/09/javaone-hotspot.html The presentation wasn't recorded but there is a video recorded from a DocklandsLJC event which is on InfoQ: https://www.infoq.com/presentations/hotspot-memory-data-structures https://www.infoq.com/presentations/hotspot-memory-data-stru...

my123 10y ago

IKVM, the Java VM for .NET converted Java bytecode to .NET since a while, and crossgen/ngen can be used for .NET AOT(Mono also has AOT)

jgalt212 10y ago

> Infrequently-used Java methods might never be compiled at all, potentially incurring a performance penalty due to repeated interpreted invocations. That sort of makes no sense. How can you incur a real performance hit if the uncompiled method is rarely called?

smitherfield 10y ago
I would assume if the method is rarely called, but very complex or time-consuming when it is called. Of course, I'm not an expert on JVMs, so I wouldn't know whether their analysis is synchronous or asynchronous or a mix of both.

premium-concern 10y ago

Is there anything this adds over Scala-native, which seems to be much further ahead already?

edko 10y ago
I think this could be a great complement to Scala-native. Right now, the project contributors have to spend effort translating the essential Java libraries that would allow Scala-native to be successful. This could really ease that job for them. It could potentially make all the Java code ever written available to Scala-native. The other thing it adds is the backing of a giant, like Oracle, which can bring stability and peace of mind to some people, when deciding whether to adopt the technology or not.
- premium-concern 10y ago
  I think this assumes that Oracle - can ship something in time - and that it will be generally available for developers (looking at how hard Oracle pushes their Java department to invent commercial features they can sell, I'm not sure about that) Looking at it, I assume that this will go the way of GWT ... not starting from "how can we make Java a good citizen in this new ecosystem?", but "here we have 100% of Java, the JDK and the JVM ... how can we compile this with full fidelity into X?".