12 ms·

It's not so much the language as it is the virtual machine / runtime environment. The JVM is one of the most solid and heavily invested systems in the history

It's not so much the language as it is the virtual machine / runtime environment. The JVM is one of the most solid and heavily invested systems in the history of business software. Notable companies who started off with PHP or Python or what have you, reached a stage in their growth where they had to either migrate, or else write their own in-house compilers to fork those languages and make them scalable. No one's ever had the need to do this with Java (although different vendors do produce their own tweaked builds in order to compete in the support space). People generally assume that a virtual machine is a disadvantage over AOT compilation. And for the "To Do List" apps that so many students and hobbyists on these forums are writing, they are correct. But JIT compilation is vastly superior for long-running server side business processes. Which is the niche that Java owns (facing some competition from .NET, which works in the same manner). It's not sexy or beginner-friendly, so the level of discussion that it gets here is disproportionately low.

dec0dedab0de 4y ago
JIT compilation is vastly superior for long-running server side business processes. How is that possible? Assuming all things are equal AOT should always be better. The advantage of a virtual machine is being able to distribute the same files to different processors and operating systems. The advantage of a JIT is that it makes your virtual machine faster.... oh how I wish Python would have an official JIT, sigh. But back to my question, if a JIT compiler takes bytecode and converts it to native code at runtime, and an AOT compiler takes source code and converts it to native code before being distributed, then how could JIT be faster? Is it that many people use AOT compilers in a generalized way instead of taking advantage of hardware specific optimizations? Or is the Java JIT compiler just so good at optimizing java byte code that it doesn't matter if it's compiled ahead of time or not? Edit: I got too many answers, but thank you all. I didn't consider the extra optimizations possible with runtime data. Please keep replying if you think of other reasons.
- peter_lawrey 4y ago
  Something not generally mentioned is that Java can inline what would be virtual methods in C++, it can also remove them as behaviour changes or code is unloaded.
- mping 4y ago
  Mostly due to profiling, the JIT gets informed on actual program execution. I wouldn't word it as vastly superior, because if you have a JIT you're probably doing some kind of tradeoff (most of the times, memory usage?), but the JIT can optimize based on the runtime profile, where an AOT't program cannot.
- vips7L 4y ago
  The JIT compiler has live runtime data on how the program is actually used and it optimizes around this instead of what the source code says. For instance, the JIT compiler can see that a function has never been passed a null object and optimize out null checks and other branches that are never taken at runtime. An AOT compiler doesn't have this information unless its provided PGO data at compile time.
  - hbrn 4y ago
    Nitpicking here. > JIT compiler can see that a function has never been passed a null object It cannot. It can only see that function has never been passed a null object so far. And the cost of such optimization might be that edge cases are way slower than anticipated. But what if those edge cases are actually the only thing that provide value for your business? Imagine you're building a monitoring system that alerts very rarely, but reaction time is really critical for you. Do you really want to JIT compiler to optimize the alerting branch out because it hasn't seen any alerts yet? My bet is that JIT compiler rarely does those optimizations by default, because it cannot know which branches in your code contain business value. I don't need my code to be twice as fast on average but twice as slow on critical branches. Now, what would be great if profiler would make suggestions about how to modify your code for performance based on usage. But those should be approved and commented by humans, because humans know whether those are valuable optimizations or a fad.
    - vips7L 4y ago
      > so far This is truly a nit pick. > Imagine you're building a monitoring system that alerts very rarely, but reaction time is really critical for you. Do you really want to JIT compiler to optimize the alerting branch out because it hasn't seen any alerts yet? Yes I do, because once the constraints are invalidated the the runtime system will instruct the JIT to recompile the code with the now uncommon branch [0]. [0] https://shipilev.net/jvm/anatomy-quarks/29-uncommon-traps/ https://shipilev.net/jvm/anatomy-quarks/29-uncommon-traps/
      - hbrn 4y ago
- ElevenLathe 4y ago
  A JIT can see the actual values that occur in hot loops, among other things, which aren't accessible at compile time but are at runtime. There is simply more information available to a JIT.
- sudarshnachakra 4y ago
  > How is that possible? Assuming all things are equal AOT should always be better. The primary thing here is that the hotter the code path the more optimized your code will be using a JIT (albeit compiled with a compiler which is slower) which is impossible with AOT (since we have a static binary compiled with -O2 or -O3 and that's it) also Java can take away the virtual dispatch if it finds a single implementation of interface or single concrete class of an abstract class which is not possible with c++ (where-in we'll always go thru the vtable which almost always resolves to a cache miss). So c++ gives you the control to choose if you want to pay the cost and if you want to pay the cost you always pay for it, but in java the runtime can be smart about it. Essentially it boils down to runtime vs compile time optimizations - runtime definitely has a richer set of profiles & patterns to make a decision and hence can be faster by quite a bit.
  - ehvatum 4y ago
    > Java can take away the virtual dispatch if it finds a single implementation of interface or single concrete class of an abstract class which is not possible with c++ (where-in we'll always go thru the vtable which almost always resolves to a cache miss) C++ LTO makes devirtualization practical to the point that your browser is benefitting from it right now: https://news.ycombinator.com/item?id=17504370 https://news.ycombinator.com/item?id=17504370
  - Someone 4y ago
    > Java can take away the virtual dispatch if it finds a single implementation of interface or single concrete class of an abstract class It can even do that if there are multiple implementations, by adding checks to detect when it is necessary to recompile the code with looser assumptions, or by keeping around multiple versions of the compiled code. It needs the ability to recompile code as assumptions are violated anyways, as “there’s only one implementation of this interface” can change when new JARs are loaded.
- blandflakes 4y ago
  Profile-guided optimization is considered pretty effective: https://en.wikipedia.org/wiki/Profile-guided_optimization https://en.wikipedia.org/wiki/Profile-guided_optimization One example of an optimization that cannot be performed AOT is anything that can be done in response to "noticing" that every time you get a List, you actually have an ArrayList. AOT can't make the assumption, but a JIT can start doing things like specializing/inlining code the implementation in question.
  - im3w1l 4y ago
    Let's consider the context here. HFT for advanced programmers. A sufficiently smart programmer working on the hot path of some HFT trading program in an AOT language can change the parameter type to ArrayList.
    - jfengel 4y ago
      Sure. There's nothing Java can do that can't be done by hand. The question is, how much effort are you going to put into the profiling to determine that that's an optimization worth making? The larger the program, the more difficult such optimizations are to find -- the example given here was a trivial one. You could do all of this by writing machine code, or constructing a programmable logic array for it. But would you actually get more trading done that way, or would be get more profit turning your programmers onto other tasks rather than one that can be handled by a machine?
      - Someone 4y ago
        > Sure. There's nothing Java can do that can't be done by hand. in practice there is. For example, the JVM can inline virtual calls to code that was loaded at runtime (even if that’s third party code for which you don’t have the source code). To do that by hand, you more or less would have to write something like the JVM yourself.
    - nostrademons 4y ago
      You can't if the method is in the standard library, or really any third-party code you don't control. That's one of the biggest advantages of profile-guided optimization that works at the bytecode/IR level. It can recognize that generic libraries are only ever called with a single concrete type, and then specialize them to that type, and then inline the data representations of the concrete data types you actually pass, and then inline all the accessors that access that data. To do that AOT you need to rewrite the library, which usually defeats the purpose of having libraries in the first place.
- kjeetgill 4y ago
  I know you're getting bombarded with a dozen answers but I don't think anyone's also mentioned speculative optimization/deoptimization. Jit can and often does assume things that can't be proven like a specific call site can be inlined through a virtual call because in practice it's always the same hard type. It can back out the optimization later if that proves to be untrue. Or if a function is always called with the same variable because it was read from a config: it can assume that it won't change.
- kaba0 4y ago
  A JIT compiler can elide a virtual method to a simple static call based on how many implementing classes are loaded at runtime, or can remove conditional branches from code (e.g. think of a setting whose boolean value is checked in a hot loop (for example something graphics related) - a JIT compiler will remove the conditional based on the setting used). Fun fact, the linux kernel actually has something similar with self-modifying code, that will remove a conditional at runtime.
- pron 4y ago
  > how could JIT be faster? 1. AOT compilers have less information about the program than JITs (they only see the program as code; JITs see the code and the current execution profile) 2. AOT compilers can only make optimisations that they can prove don't change semantics, while JITs can be more aggressive, making optimisations that are speculative, deoptimising if wrong.

dec0dedab0de 4y ago

JIT compilation is vastly superior for long-running server side business processes. How is that possible? Assuming all things are equal AOT should always be better. The advantage of a virtual machine is being able to distribute the same files to different processors and operating systems. The advantage of a JIT is that it makes your virtual machine faster.... oh how I wish Python would have an official JIT, sigh. But back to my question, if a JIT compiler takes bytecode and converts it to native code at runtime, and an AOT compiler takes source code and converts it to native code before being distributed, then how could JIT be faster? Is it that many people use AOT compilers in a generalized way instead of taking advantage of hardware specific optimizations? Or is the Java JIT compiler just so good at optimizing java byte code that it doesn't matter if it's compiled ahead of time or not? Edit: I got too many answers, but thank you all. I didn't consider the extra optimizations possible with runtime data. Please keep replying if you think of other reasons.

peter_lawrey 4y ago
Something not generally mentioned is that Java can inline what would be virtual methods in C++, it can also remove them as behaviour changes or code is unloaded.
mping 4y ago
Mostly due to profiling, the JIT gets informed on actual program execution. I wouldn't word it as vastly superior, because if you have a JIT you're probably doing some kind of tradeoff (most of the times, memory usage?), but the JIT can optimize based on the runtime profile, where an AOT't program cannot.
vips7L 4y ago
The JIT compiler has live runtime data on how the program is actually used and it optimizes around this instead of what the source code says. For instance, the JIT compiler can see that a function has never been passed a null object and optimize out null checks and other branches that are never taken at runtime. An AOT compiler doesn't have this information unless its provided PGO data at compile time.
- hbrn 4y ago
  Nitpicking here. > JIT compiler can see that a function has never been passed a null object It cannot. It can only see that function has never been passed a null object so far. And the cost of such optimization might be that edge cases are way slower than anticipated. But what if those edge cases are actually the only thing that provide value for your business? Imagine you're building a monitoring system that alerts very rarely, but reaction time is really critical for you. Do you really want to JIT compiler to optimize the alerting branch out because it hasn't seen any alerts yet? My bet is that JIT compiler rarely does those optimizations by default, because it cannot know which branches in your code contain business value. I don't need my code to be twice as fast on average but twice as slow on critical branches. Now, what would be great if profiler would make suggestions about how to modify your code for performance based on usage. But those should be approved and commented by humans, because humans know whether those are valuable optimizations or a fad.
  - vips7L 4y ago
    > so far This is truly a nit pick. > Imagine you're building a monitoring system that alerts very rarely, but reaction time is really critical for you. Do you really want to JIT compiler to optimize the alerting branch out because it hasn't seen any alerts yet? Yes I do, because once the constraints are invalidated the the runtime system will instruct the JIT to recompile the code with the now uncommon branch [0]. [0] https://shipilev.net/jvm/anatomy-quarks/29-uncommon-traps/ https://shipilev.net/jvm/anatomy-quarks/29-uncommon-traps/
    - hbrn 4y ago
ElevenLathe 4y ago
A JIT can see the actual values that occur in hot loops, among other things, which aren't accessible at compile time but are at runtime. There is simply more information available to a JIT.
sudarshnachakra 4y ago
> How is that possible? Assuming all things are equal AOT should always be better. The primary thing here is that the hotter the code path the more optimized your code will be using a JIT (albeit compiled with a compiler which is slower) which is impossible with AOT (since we have a static binary compiled with -O2 or -O3 and that's it) also Java can take away the virtual dispatch if it finds a single implementation of interface or single concrete class of an abstract class which is not possible with c++ (where-in we'll always go thru the vtable which almost always resolves to a cache miss). So c++ gives you the control to choose if you want to pay the cost and if you want to pay the cost you always pay for it, but in java the runtime can be smart about it. Essentially it boils down to runtime vs compile time optimizations - runtime definitely has a richer set of profiles & patterns to make a decision and hence can be faster by quite a bit.
- ehvatum 4y ago
  > Java can take away the virtual dispatch if it finds a single implementation of interface or single concrete class of an abstract class which is not possible with c++ (where-in we'll always go thru the vtable which almost always resolves to a cache miss) C++ LTO makes devirtualization practical to the point that your browser is benefitting from it right now: https://news.ycombinator.com/item?id=17504370 https://news.ycombinator.com/item?id=17504370
- Someone 4y ago
  > Java can take away the virtual dispatch if it finds a single implementation of interface or single concrete class of an abstract class It can even do that if there are multiple implementations, by adding checks to detect when it is necessary to recompile the code with looser assumptions, or by keeping around multiple versions of the compiled code. It needs the ability to recompile code as assumptions are violated anyways, as “there’s only one implementation of this interface” can change when new JARs are loaded.
blandflakes 4y ago
Profile-guided optimization is considered pretty effective: https://en.wikipedia.org/wiki/Profile-guided_optimization https://en.wikipedia.org/wiki/Profile-guided_optimization One example of an optimization that cannot be performed AOT is anything that can be done in response to "noticing" that every time you get a List, you actually have an ArrayList. AOT can't make the assumption, but a JIT can start doing things like specializing/inlining code the implementation in question.
- im3w1l 4y ago
  Let's consider the context here. HFT for advanced programmers. A sufficiently smart programmer working on the hot path of some HFT trading program in an AOT language can change the parameter type to ArrayList.
  - jfengel 4y ago
    Sure. There's nothing Java can do that can't be done by hand. The question is, how much effort are you going to put into the profiling to determine that that's an optimization worth making? The larger the program, the more difficult such optimizations are to find -- the example given here was a trivial one. You could do all of this by writing machine code, or constructing a programmable logic array for it. But would you actually get more trading done that way, or would be get more profit turning your programmers onto other tasks rather than one that can be handled by a machine?
    - Someone 4y ago
      > Sure. There's nothing Java can do that can't be done by hand. in practice there is. For example, the JVM can inline virtual calls to code that was loaded at runtime (even if that’s third party code for which you don’t have the source code). To do that by hand, you more or less would have to write something like the JVM yourself.
  - nostrademons 4y ago
    You can't if the method is in the standard library, or really any third-party code you don't control. That's one of the biggest advantages of profile-guided optimization that works at the bytecode/IR level. It can recognize that generic libraries are only ever called with a single concrete type, and then specialize them to that type, and then inline the data representations of the concrete data types you actually pass, and then inline all the accessors that access that data. To do that AOT you need to rewrite the library, which usually defeats the purpose of having libraries in the first place.
kjeetgill 4y ago
I know you're getting bombarded with a dozen answers but I don't think anyone's also mentioned speculative optimization/deoptimization. Jit can and often does assume things that can't be proven like a specific call site can be inlined through a virtual call because in practice it's always the same hard type. It can back out the optimization later if that proves to be untrue. Or if a function is always called with the same variable because it was read from a config: it can assume that it won't change.
kaba0 4y ago
A JIT compiler can elide a virtual method to a simple static call based on how many implementing classes are loaded at runtime, or can remove conditional branches from code (e.g. think of a setting whose boolean value is checked in a hot loop (for example something graphics related) - a JIT compiler will remove the conditional based on the setting used). Fun fact, the linux kernel actually has something similar with self-modifying code, that will remove a conditional at runtime.
pron 4y ago
> how could JIT be faster? 1. AOT compilers have less information about the program than JITs (they only see the program as code; JITs see the code and the current execution profile) 2. AOT compilers can only make optimisations that they can prove don't change semantics, while JITs can be more aggressive, making optimisations that are speculative, deoptimising if wrong.