4 ms·

I'm not familiar enough with compilers, but why would an ahead of time compiler perform worse than a just in time compiler in a static language? I think I'd und

by avbor 10y ago

I'm not familiar enough with compilers, but why would an ahead of time compiler perform worse than a just in time compiler in a static language? I think I'd understand if it was a dynamic language, because you can't know the types for sure until you start running the program, but are similar issues present for Java?

Scarbutt 10y ago
A JIT can use information from runtime for optimizations, you pay a cost in startup time. Being worse or better depends on the workloads and implementations.
- CalChris 10y ago
  AOT can as well. PGO.
  - alblue 10y ago
    There are a set of optimisations that you can only do at runtime that you can't do with AOT. Anything that depends on data is something that you can't reliably do, such as eliding null checks if you can prove this cannot happen based on the data passed in. There are also cases where you can have multiple subclasses of a type such as a normal and a debug subclass, or multiple drivers for different back ends such as MongoDB or Cassandra, only one of which is used at runtime but you cannot know ahead of time which is selected (for example, it's based on an environment variable or system property). The point is that while AOT can do a set of optimisations, including whole module analysis, there are a set which are only available at runtime.
  - vidarh 10y ago
    Yes, but that locks you in to optimising for whatever you covered with your profiling. If the character of your data changes, it'll take a recompile to change how the application performs, while a JIT can potentially choose to deoptimise/reoptimise. I'm all for "as static as possible" toolchains, but there are optimization opportunities you simply won't have with AOT, PGO or not. E.g. consider something trivial: A program doing certain image operations that depends on dimensions passed in on the command line. A JIT could optimise the inner loops for the actual operations. To get the same with AOT even with PGO would be totally unable to deal with it without causing a massive explosion in code size.
    - pjmlp 10y ago
      Hence why .NET also has MPGO, Managed Profile Guided Optimizations for NGEN. https://msdn.microsoft.com/en-us/library/hh873180(v=vs.110).aspx https://msdn.microsoft.com/en-us/library/hh873180(v=vs.110)....
    - srean 10y ago
      In theory yes, but sadly i have never seen that promise realized in any consistent fashion. Same goes for Java's escape analysis. Although the principle is sound i think the engineering required is horrendously difficult to make it robust. In a very narrow window of variation it works, but should you step out of that zone it fails pretty badly. I think it will take many more years, till then pgo and metaprogramming it is.
      - pjmlp 10y ago
        Regarding escape analysis, beware that the OpenJDK is the worst of them all. IBM J9, Graal are much better at it, and I bet Aonix, Azul and other JDKs targeting high performance deployment scenarios are even better.
      - mike_hearn 10y ago
        Speculative optimisations yield about a 20% improvement in Java and more for higher level languages like Scala (and for Ruby it's off the charts). HotSpot isn't perfect by any means and C2's EA is not strong enough, but it's on track to be replaced with Graal (which is a part of what AOT is all about - you don't want your VM compiling itself at the same time as compiling your app).
  - mike_hearn 10y ago
    PGO in C++ can yield quite significant speedups, however the difficulty of integrating it into a development workflow means that in practice it's hardly ever used. One of Java's accomplishments is that it brought PGO to the masses by making it entirely built in and automatic.
yuriks 10y ago
Yes, Java code relies a lot on devirtualization to get good performance. This is because the language allows every function to be virtual by default. The flip-side is that this make coding styles that make heavy usage of interfaces (arguably good for testing, interoperability, decoupling, etc.) effectively "free" compared to ones using concrete classes.
psuter 10y ago
One thing JIT compilers are good at is specializing methods for common (runtime) types; e.g. you have a method operating on Iterable but it turns out most of the time you get a List as input; you can generate code that bypasses the method lookups for .size etc. The question of whether this pays off is usually only answerable at runtime.
- bad_user 10y ago
  Actually that's not a thing most JIT compilers are good at, as such runtimes are hard to build. For the mainstream ones, you only have Java and the major Javascript runtimes.
h4nkoslo 10y ago
Most of the JIT optimizations amount to "it's been called like this in the past; assume it'll always be & depotimize (at a penalty) if it isn't." That potentially includes the fully resolved types of objects (ie devirtualization), branch prediction (stronger than the CPU can do; for instance, if a value is only used inside a branch that's never taken, don't bother mutating it), data sizes (this "array" is only ever size 2, store it in registers), dead code elimination (keeps the compiled code small), and a whole bunch more fun stuff.
- catnaroek 10y ago
  > assume it'll always be & depotimize (at a penalty) if it isn't Stuff like this makes me nervous. Performance is already a complex topic, and stuff like this makes it even more complex. Unnecessarily so. If we were talking about a very high-level programming language (say, Prolog), you could argue that the expressiveness benefits outweigh the cost of the runtime system's complexity. But Java isn't even as expressive as C++, let alone Prolog. > fully resolved types of objects (ie devirtualization) C++ (and similar languages: D, Rust, etc.) and MLton (a Standard ML implementation) have been using monomorphization for ages, which is a compile-time analogue of devirtualization. Moreover, monomorphization has important advantages over devirtualization: (0) It's completely predictable. You don't need to guess when it will happen. It happens iff the concrete type (and its relevant vtables, if necessary) can be determined at compile-time: https://blog.rust-lang.org/2015/05/11/traits.html https://blog.rust-lang.org/2015/05/11/traits.html (1) It's always a sound optimization, so it doesn't have to be undone at runtime under any circumstances. (2) It's relatively simple to implement. In fact, a compiler front-end can completely monomorphize a program before handing it over to the back-end for target code generation. > if a value is only used inside a branch that's never taken, don't bother mutating it) The best way to handle unreachable branches is to avoid creating them in the first place. With proper use of algebraic data types and pattern matching, unreachable branches can be kept to a minimum, or even outright eliminated in many cases. > data sizes (this "array" is only ever size 2, store it in registers) C and similar languages natively handle statically sized arrays, so there's no need for runtime profiling and analysis just to determine that an array will always have size 2. ML does something even better: you just use tuples (in this case, pairs), which reflect your intent much better than using arrays whose size has to be tested or guessed. --- What I take away from this is that the JVM's supposedly “fancy” optimizations exist primarily to work around the Java language's lack of amenability to static analysis.
mike_hearn 10y ago
Almost all languages can benefit from profile guided optimisations, including very static languages like C++

by avbor 10y ago

Scarbutt 10y ago

A JIT can use information from runtime for optimizations, you pay a cost in startup time. Being worse or better depends on the workloads and implementations.

CalChris 10y ago
AOT can as well. PGO.
- alblue 10y ago
  There are a set of optimisations that you can only do at runtime that you can't do with AOT. Anything that depends on data is something that you can't reliably do, such as eliding null checks if you can prove this cannot happen based on the data passed in. There are also cases where you can have multiple subclasses of a type such as a normal and a debug subclass, or multiple drivers for different back ends such as MongoDB or Cassandra, only one of which is used at runtime but you cannot know ahead of time which is selected (for example, it's based on an environment variable or system property). The point is that while AOT can do a set of optimisations, including whole module analysis, there are a set which are only available at runtime.
- vidarh 10y ago
  Yes, but that locks you in to optimising for whatever you covered with your profiling. If the character of your data changes, it'll take a recompile to change how the application performs, while a JIT can potentially choose to deoptimise/reoptimise. I'm all for "as static as possible" toolchains, but there are optimization opportunities you simply won't have with AOT, PGO or not. E.g. consider something trivial: A program doing certain image operations that depends on dimensions passed in on the command line. A JIT could optimise the inner loops for the actual operations. To get the same with AOT even with PGO would be totally unable to deal with it without causing a massive explosion in code size.
  - pjmlp 10y ago
    Hence why .NET also has MPGO, Managed Profile Guided Optimizations for NGEN. https://msdn.microsoft.com/en-us/library/hh873180(v=vs.110).aspx https://msdn.microsoft.com/en-us/library/hh873180(v=vs.110)....
  - srean 10y ago
    In theory yes, but sadly i have never seen that promise realized in any consistent fashion. Same goes for Java's escape analysis. Although the principle is sound i think the engineering required is horrendously difficult to make it robust. In a very narrow window of variation it works, but should you step out of that zone it fails pretty badly. I think it will take many more years, till then pgo and metaprogramming it is.
    - pjmlp 10y ago
      Regarding escape analysis, beware that the OpenJDK is the worst of them all. IBM J9, Graal are much better at it, and I bet Aonix, Azul and other JDKs targeting high performance deployment scenarios are even better.
    - mike_hearn 10y ago
      Speculative optimisations yield about a 20% improvement in Java and more for higher level languages like Scala (and for Ruby it's off the charts). HotSpot isn't perfect by any means and C2's EA is not strong enough, but it's on track to be replaced with Graal (which is a part of what AOT is all about - you don't want your VM compiling itself at the same time as compiling your app).
- mike_hearn 10y ago
  PGO in C++ can yield quite significant speedups, however the difficulty of integrating it into a development workflow means that in practice it's hardly ever used. One of Java's accomplishments is that it brought PGO to the masses by making it entirely built in and automatic.

yuriks 10y ago

Yes, Java code relies a lot on devirtualization to get good performance. This is because the language allows every function to be virtual by default. The flip-side is that this make coding styles that make heavy usage of interfaces (arguably good for testing, interoperability, decoupling, etc.) effectively "free" compared to ones using concrete classes.

psuter 10y ago

One thing JIT compilers are good at is specializing methods for common (runtime) types; e.g. you have a method operating on Iterable but it turns out most of the time you get a List as input; you can generate code that bypasses the method lookups for .size etc. The question of whether this pays off is usually only answerable at runtime.

bad_user 10y ago
Actually that's not a thing most JIT compilers are good at, as such runtimes are hard to build. For the mainstream ones, you only have Java and the major Javascript runtimes.

h4nkoslo 10y ago

Most of the JIT optimizations amount to "it's been called like this in the past; assume it'll always be & depotimize (at a penalty) if it isn't." That potentially includes the fully resolved types of objects (ie devirtualization), branch prediction (stronger than the CPU can do; for instance, if a value is only used inside a branch that's never taken, don't bother mutating it), data sizes (this "array" is only ever size 2, store it in registers), dead code elimination (keeps the compiled code small), and a whole bunch more fun stuff.

catnaroek 10y ago
> assume it'll always be & depotimize (at a penalty) if it isn't Stuff like this makes me nervous. Performance is already a complex topic, and stuff like this makes it even more complex. Unnecessarily so. If we were talking about a very high-level programming language (say, Prolog), you could argue that the expressiveness benefits outweigh the cost of the runtime system's complexity. But Java isn't even as expressive as C++, let alone Prolog. > fully resolved types of objects (ie devirtualization) C++ (and similar languages: D, Rust, etc.) and MLton (a Standard ML implementation) have been using monomorphization for ages, which is a compile-time analogue of devirtualization. Moreover, monomorphization has important advantages over devirtualization: (0) It's completely predictable. You don't need to guess when it will happen. It happens iff the concrete type (and its relevant vtables, if necessary) can be determined at compile-time: https://blog.rust-lang.org/2015/05/11/traits.html https://blog.rust-lang.org/2015/05/11/traits.html (1) It's always a sound optimization, so it doesn't have to be undone at runtime under any circumstances. (2) It's relatively simple to implement. In fact, a compiler front-end can completely monomorphize a program before handing it over to the back-end for target code generation. > if a value is only used inside a branch that's never taken, don't bother mutating it) The best way to handle unreachable branches is to avoid creating them in the first place. With proper use of algebraic data types and pattern matching, unreachable branches can be kept to a minimum, or even outright eliminated in many cases. > data sizes (this "array" is only ever size 2, store it in registers) C and similar languages natively handle statically sized arrays, so there's no need for runtime profiling and analysis just to determine that an array will always have size 2. ML does something even better: you just use tuples (in this case, pairs), which reflect your intent much better than using arrays whose size has to be tested or guessed. --- What I take away from this is that the JVM's supposedly “fancy” optimizations exist primarily to work around the Java language's lack of amenability to static analysis.

mike_hearn 10y ago

Almost all languages can benefit from profile guided optimisations, including very static languages like C++