6 ms·

To the best of my knowledge, GraalVM native images are probably slower at runtime, due to the hotspot vm being unavailable.

smabie 6y ago
GraalVM native images are slower right now, but there's no reason that has to be the case in the future. I think we all have realized over the last 2 decades that though JIT compilers (like the JVM) should be theoretically faster than a traditional native code compiler, this is rarely (if ever) the case. I wouldn't be surprised if over the next 10 years most JVM code (including Java, Scala, etc) is compiled to native images. It's both clear that Oracle is pushing GraalVM hard and that it's what the consumer wants. If performance was identical, I can't think of a single reason why anyone would choose the regular JDK over GraalVM. It's not like people are slinging jars over the network between linux and windows... hell, now with WSL, that's not even a good reason either.
- rlp 6y ago
  HotSpot uses a JIT compiler that dynamically recompiles code at runtime to speed up hot code paths. GraalVM native compilation cannot do this, and given the nature of Java's non-reified generics, it's very hard to get equivalent speeds at runtime. Profile-guided optimization is the only hope to recover some of that performance gap, and it's currently only available in GraalVM EE. I'm not sure how the benchmarks compare to HotSpot.
  - smabie 6y ago
    Why would type erasure hurt performance? Certainly expressiveness is curtailed due to Java's generics design (or lack thereof), but I wasn't aware of any performance implications.
    - erik_seaberg 6y ago
      Whenever a generic class method returns an erased type and the caller tries to use it, javac has to insert a runtime check (cast) so the bytecode is typesafe. It also forces boxing of primitive types.
      - smabie 6y ago
        But since the type of a function can't change at runtime, why is this necessary?
        erik_seaberg 6y ago
        Because of erasure, there is only one copy of List#get(int), and it returns Object (because there's no other upper bound). The source might say List<String> l = ... String s = l.get(0); which looks typesafe, but to pass the verifier and load, the bytecode has to accept an Object and cast it to String before using it, just like Java programmers had to write by hand before generics (this really sucked). There's not even anything stopping you from putting non-Strings in that List instance via reflection or generic casts (if you ignore the warning) or using a really old javac. You could use a checked collection but that just adds a runtime check during writes that ensures the runtime check during reads will never fail (but the bytecode verifier still won't let you remove it!)
        erik_seaberg 6y ago
        I should have written ArrayList#get. It's possible to write a class that implements List<String> in a typesafe way, but callers using the standard List interface receive an Object and have to downcast.
  - doctor_eval 6y ago
    If the above RAM figures are correct (⅕ ram for native), and of JIT performance is 20% faster than native, then I’m pretty sure that it’s cheaper to add 20% more CPU than 5x more RAM. Especially now that we are talking Gravitons and chiplets.
- tom_mellior 6y ago
  > I think we all have realized over the last 2 decades that though JIT compilers (like the JVM) should be theoretically faster than a traditional native code compiler, this is rarely (if ever) the case. This is a very broad claim. Do you have data to support it? It would mean comparing production-quality JIT and AOT compilers for the same language, which are hard to find. I'm vaguely aware that LLVM's JIT ambitions never worked out because compile times were too slow, which (IIRC) they tried to mitigate by running fewer optimizations in JIT mode, which meant the quality of the generated code wasn't as good as AOT. I wouldn't call this production quality though, and even so it wouldn't allow you to make your claim as broadly as you are.
- alblue 6y ago
  Actually the JIT compiler often outperforms native code when hot because the latter doesn’t typically get compiled with PGO and the JIT is effectively PGO at runtime. In addition, the JIT can perform optimisations that a native compiler can’t do, because the JIT can take different decisions if a field is null repeatedly, and thus elide the inclusion of code that supports the non null case leading to potential further optimisations due to dead code elimination and additional inlining opportunities. This is something that PGO won’t give you – the native code compiler will have to be defensive and compile everything, not just what it measured on a few runs of test cases. In the JIT’s case, when an assumption is violated, it falls back to the interpreter and recompiles with the new knowledge of the change and thus includes more code than before. The speed advantages of the JIT tend not to be seen on comparison sites like Debian’s language shootout because they don’t exclude start up time and rarely run the code a sufficient number of times to trigger the JIT’s compilation (10,000 method calls by default) to become hot. It’s why the JVM world has tools like JMH to do micro benchmarking. Finally in the Graal case, it’s actually possible to use the Graal compiler as the hotspot replacement so you can see the difference in speed and behaviour between the two. Graal has some nice wins but also some disadvantages; it’s good at fixing some of Scales’s generated code which is why Twitter are big fans, but on regular Java code the advantages are not as pronounced. The main benefit to Graal is the reduction in startup time, not execution speed at runtime. For long running servers you are better off with the JVM’s JIT for performance; for short start up times (like AWS Lambda) you could be better off with jaotc or Graal.
  - pjmlp 6y ago
    And if using a recent Java version (meaning not 8), that JIT code is cached between executions and can even take PGO data into account. There are a couple of talks from the .NET team regarding the same issue, and how they are planning that for full support of all .NET features, the way forward is a mix of AOT/JIT. Android team also learned the hard way that AOT only wasn't as good idea as they thought, hence why ART now does all combinations, interpreted, JIT and AOT with PGO and code cache.
- moonchild 6y ago
  It's not very easy to compare java's performance with that of other compiled languages, because of the GC. The closest native analogue is go, which is much slower than java, although it's not quite a fair comparison since go uses a concurrent GC by default and java does not. Java is quite fast for what it does, and I don't expect any runtime to make it significantly faster (aside the fact that java isn't slow, and its performance problems are memory usage and lack of value types, which native vs. JIT has no impact on).
  - smabie 6y ago
    Languages like Ocaml and Haskell are both around Java speed, though. And even though both languages have more primitive GCs than Java, in practice, they tend to use less memory and have lower latency. I can only think of one situation when using Ocaml that I had to think about the GC (I was allocated huge floating point arrays), while I can't even count the number of times I had to deal with a runaway heap or unacceptably long pauses. The last JVM version I've really used was 7, though. Perhaps things have gotten better?
    - jerven 6y ago
      Yes, things have gotten a lot better. G1GC is the default now and we have free options like ZGC and ShenandoahGC for low latency options. Of course there are the IBM options and Azul as well. Scalar replacement got better, and with Graal partial evaluation gives reduced heap allocation rates. Which can have big impacts. Future is even more bright with inline (value) types going towards even less allocations and better memory layouts.
      - grashalm 6y ago
        You mean partial escape analysis and scalar replacement for reduced heap allocation rates. Partial evaluation is something else, also pretty useful but for other reasons.
        jerven 6y ago
- pjmlp 6y ago
  AOT compilation doesn't make good use of PGO data (you need a very good test data set) and there is zero chance of inlining across dynamic libraries. What most developers want is not having to pay for their tools, hence why so much love for GraalVM community edition. Some form of AOT compilation has been part of commercial JDKs since early 2000's. In fact, the JIT cache that is now also available for free on OpenJDK traces back to J/Rockit and similarly the JIT code caches now available on OpenJ9 go all the way back to WebSphere Real Time JVM. Also this is another point, by holding on to Java 8, those JIT caches are not available to those that only want the free JDK variants. However it is a warm feeling that despite the critic, here is a project that they have kept aliven from Sun Labs (aka MaximeVM) and not only have brought it to production, they have a long term roadmap to replace C2 with it (Project Metropolis).
  - geodel 6y ago
    > What most developers want is not having to pay for their tools, hence why so much love for GraalVM community edition. Well I agree that developers do not want to pay in general. But Graal VM CE/EE are firmly in enterprise domain, so companies have to take a call on Graal.
    - pjmlp 6y ago
      Graal VM CE is free beer, hence the attention it is getting now.
- thu2111 6y ago
  It's not like people are slinging jars over the network between linux and windows That happens all the time. Examples: - Anyone using IDE plugins to IntelliJ, PyCharm, WebStorm or any other developer IDE - Any team where developers are running a mix of operating systems but work on a shared Java codebase - Any place developers pull in Java dependencies from Maven Central/JCenter and then use them in an Android app
- hrgiger 6y ago
  Debugging native image would be well utilized as well
geodel 6y ago
Indeed. And when documents say "Unless low memory usage is top requirement just use regular JDK for better performance." It does not give much confidence in native solution.
- topspin 6y ago
  Given Java's outrageous memory usage I think this is pretty compelling benefit. It is not easy to provision resources for Java applications relative to just about everything else.

smabie 6y ago

GraalVM native images are slower right now, but there's no reason that has to be the case in the future. I think we all have realized over the last 2 decades that though JIT compilers (like the JVM) should be theoretically faster than a traditional native code compiler, this is rarely (if ever) the case. I wouldn't be surprised if over the next 10 years most JVM code (including Java, Scala, etc) is compiled to native images. It's both clear that Oracle is pushing GraalVM hard and that it's what the consumer wants. If performance was identical, I can't think of a single reason why anyone would choose the regular JDK over GraalVM. It's not like people are slinging jars over the network between linux and windows... hell, now with WSL, that's not even a good reason either.

rlp 6y ago
HotSpot uses a JIT compiler that dynamically recompiles code at runtime to speed up hot code paths. GraalVM native compilation cannot do this, and given the nature of Java's non-reified generics, it's very hard to get equivalent speeds at runtime. Profile-guided optimization is the only hope to recover some of that performance gap, and it's currently only available in GraalVM EE. I'm not sure how the benchmarks compare to HotSpot.
- smabie 6y ago
  Why would type erasure hurt performance? Certainly expressiveness is curtailed due to Java's generics design (or lack thereof), but I wasn't aware of any performance implications.
  - erik_seaberg 6y ago
    Whenever a generic class method returns an erased type and the caller tries to use it, javac has to insert a runtime check (cast) so the bytecode is typesafe. It also forces boxing of primitive types.
    - smabie 6y ago
      But since the type of a function can't change at runtime, why is this necessary?
      - erik_seaberg 6y ago
        Because of erasure, there is only one copy of List#get(int), and it returns Object (because there's no other upper bound). The source might say List<String> l = ... String s = l.get(0); which looks typesafe, but to pass the verifier and load, the bytecode has to accept an Object and cast it to String before using it, just like Java programmers had to write by hand before generics (this really sucked). There's not even anything stopping you from putting non-Strings in that List instance via reflection or generic casts (if you ignore the warning) or using a really old javac. You could use a checked collection but that just adds a runtime check during writes that ensures the runtime check during reads will never fail (but the bytecode verifier still won't let you remove it!)
        erik_seaberg 6y ago
        I should have written ArrayList#get. It's possible to write a class that implements List<String> in a typesafe way, but callers using the standard List interface receive an Object and have to downcast.
- doctor_eval 6y ago
  If the above RAM figures are correct (⅕ ram for native), and of JIT performance is 20% faster than native, then I’m pretty sure that it’s cheaper to add 20% more CPU than 5x more RAM. Especially now that we are talking Gravitons and chiplets.
tom_mellior 6y ago
> I think we all have realized over the last 2 decades that though JIT compilers (like the JVM) should be theoretically faster than a traditional native code compiler, this is rarely (if ever) the case. This is a very broad claim. Do you have data to support it? It would mean comparing production-quality JIT and AOT compilers for the same language, which are hard to find. I'm vaguely aware that LLVM's JIT ambitions never worked out because compile times were too slow, which (IIRC) they tried to mitigate by running fewer optimizations in JIT mode, which meant the quality of the generated code wasn't as good as AOT. I wouldn't call this production quality though, and even so it wouldn't allow you to make your claim as broadly as you are.
alblue 6y ago
Actually the JIT compiler often outperforms native code when hot because the latter doesn’t typically get compiled with PGO and the JIT is effectively PGO at runtime. In addition, the JIT can perform optimisations that a native compiler can’t do, because the JIT can take different decisions if a field is null repeatedly, and thus elide the inclusion of code that supports the non null case leading to potential further optimisations due to dead code elimination and additional inlining opportunities. This is something that PGO won’t give you – the native code compiler will have to be defensive and compile everything, not just what it measured on a few runs of test cases. In the JIT’s case, when an assumption is violated, it falls back to the interpreter and recompiles with the new knowledge of the change and thus includes more code than before. The speed advantages of the JIT tend not to be seen on comparison sites like Debian’s language shootout because they don’t exclude start up time and rarely run the code a sufficient number of times to trigger the JIT’s compilation (10,000 method calls by default) to become hot. It’s why the JVM world has tools like JMH to do micro benchmarking. Finally in the Graal case, it’s actually possible to use the Graal compiler as the hotspot replacement so you can see the difference in speed and behaviour between the two. Graal has some nice wins but also some disadvantages; it’s good at fixing some of Scales’s generated code which is why Twitter are big fans, but on regular Java code the advantages are not as pronounced. The main benefit to Graal is the reduction in startup time, not execution speed at runtime. For long running servers you are better off with the JVM’s JIT for performance; for short start up times (like AWS Lambda) you could be better off with jaotc or Graal.
- pjmlp 6y ago
  And if using a recent Java version (meaning not 8), that JIT code is cached between executions and can even take PGO data into account. There are a couple of talks from the .NET team regarding the same issue, and how they are planning that for full support of all .NET features, the way forward is a mix of AOT/JIT. Android team also learned the hard way that AOT only wasn't as good idea as they thought, hence why ART now does all combinations, interpreted, JIT and AOT with PGO and code cache.
moonchild 6y ago
It's not very easy to compare java's performance with that of other compiled languages, because of the GC. The closest native analogue is go, which is much slower than java, although it's not quite a fair comparison since go uses a concurrent GC by default and java does not. Java is quite fast for what it does, and I don't expect any runtime to make it significantly faster (aside the fact that java isn't slow, and its performance problems are memory usage and lack of value types, which native vs. JIT has no impact on).
- smabie 6y ago
  Languages like Ocaml and Haskell are both around Java speed, though. And even though both languages have more primitive GCs than Java, in practice, they tend to use less memory and have lower latency. I can only think of one situation when using Ocaml that I had to think about the GC (I was allocated huge floating point arrays), while I can't even count the number of times I had to deal with a runaway heap or unacceptably long pauses. The last JVM version I've really used was 7, though. Perhaps things have gotten better?
  - jerven 6y ago
    Yes, things have gotten a lot better. G1GC is the default now and we have free options like ZGC and ShenandoahGC for low latency options. Of course there are the IBM options and Azul as well. Scalar replacement got better, and with Graal partial evaluation gives reduced heap allocation rates. Which can have big impacts. Future is even more bright with inline (value) types going towards even less allocations and better memory layouts.
    - grashalm 6y ago
      You mean partial escape analysis and scalar replacement for reduced heap allocation rates. Partial evaluation is something else, also pretty useful but for other reasons.
      - jerven 6y ago
pjmlp 6y ago
AOT compilation doesn't make good use of PGO data (you need a very good test data set) and there is zero chance of inlining across dynamic libraries. What most developers want is not having to pay for their tools, hence why so much love for GraalVM community edition. Some form of AOT compilation has been part of commercial JDKs since early 2000's. In fact, the JIT cache that is now also available for free on OpenJDK traces back to J/Rockit and similarly the JIT code caches now available on OpenJ9 go all the way back to WebSphere Real Time JVM. Also this is another point, by holding on to Java 8, those JIT caches are not available to those that only want the free JDK variants. However it is a warm feeling that despite the critic, here is a project that they have kept aliven from Sun Labs (aka MaximeVM) and not only have brought it to production, they have a long term roadmap to replace C2 with it (Project Metropolis).
- geodel 6y ago
  > What most developers want is not having to pay for their tools, hence why so much love for GraalVM community edition. Well I agree that developers do not want to pay in general. But Graal VM CE/EE are firmly in enterprise domain, so companies have to take a call on Graal.
  - pjmlp 6y ago
    Graal VM CE is free beer, hence the attention it is getting now.
thu2111 6y ago
It's not like people are slinging jars over the network between linux and windows That happens all the time. Examples: - Anyone using IDE plugins to IntelliJ, PyCharm, WebStorm or any other developer IDE - Any team where developers are running a mix of operating systems but work on a shared Java codebase - Any place developers pull in Java dependencies from Maven Central/JCenter and then use them in an Android app
hrgiger 6y ago
Debugging native image would be well utilized as well

geodel 6y ago

Indeed. And when documents say "Unless low memory usage is top requirement just use regular JDK for better performance." It does not give much confidence in native solution.

topspin 6y ago
Given Java's outrageous memory usage I think this is pretty compelling benefit. It is not easy to provision resources for Java applications relative to just about everything else.