3 ms·

I read these JIT promises decades ago, yet AOT languages like C and Rust are still the pinnacle of performance. How come?

by k__ 4y ago

vlovich123 4y ago
PGO can be thought of in a sense as a form of JIT (just that you have to manually capture a profile and keep it up to date to be relevant vs happening transparently at runtime). Still, it can offer benefits [1]. So clearly something that automatically JITted without the headache of managing PGO profiles could make it MUCH easier to turn globally without needing to really do a lot of work. [1] https://blog.chromium.org/2020/08/chrome-just-got-faster-with-profile.html https://blog.chromium.org/2020/08/chrome-just-got-faster-wit...
pca006132 4y ago
I think part of the reason is due to the programmer: People writing in AOT languages such as C, C++ and Rust usually care more about performance and knows more about low level details that matter for performance. For example, they may try to optimize data layout to avoid cache misses, which JavaScript programmers generally won't (if they can). Also, compiling AOT can allow programmers to look at the optimization done by the compiler and determine if they need to further tweak the code to improve performance.
dathinab 4y ago
Complexity/Time Constraints, predicability, compiler-existance and programming language design. Complexity: - JIT is only faster if it often enough recognizes the right things to JIT in the right way. - At the same time the JIT compilers share resources with the running program limiting how complex the analytics and optimizations it can do can become. This is not just a problem wrt. to hurting the main applications performance due to reducing available resources but also e.g. matters for energy usage. - Worse there is often real time time bound on how long a JIT compiler can take no matter how much resources it uses or responsiveness will suffer to much. - Similar if no additional tricks are used (which often add painful complexity) the optimizations have to be done every time the program is started again and again, hurting auto scaling or ephemeral programs. - There is also the problem that workload/data specific optimizations (which gives JIT the additional edge) rely on some data/workloads always having a specific format. If this invariants change they might become slow or might even to fallback to other code (which implies the invariant also needed to be checked). One exception is if you hint to the compiler that certain invariants (e.g. function pointers) won't change (like in the OP article). Predictability: - the performance of AOT focused language code is often more predictable making it often easier to optimize - in JIT languages you rely more on the compiler being able to detect the right patterns, this isn't always easy and even with AOT languages this sometimes isn't easy even through the compiler can do much more work to analyze the context compiler-existance: - Many AOT language can take advantage of LLVM/GCC so a lot of work has gone into optimizing them before they even started. - And while you can use LLVM to JIT it is often not viable as it's a rather large dependency with a lot of different versions which itself is optimized for throughput and not latency and is also tuned for a use-case which is allowed to take a while and consume quite a bit of resources. Language Design: - Features AOT languages are often designed with their AOT nature in mind, many JIT/Interpreted languages are not. Sometimes they outright have features which can impede JIT optimizations or at least make them more complicated/harder to find/less fast. - If you throw enough money at a badly for JIT designed language you probably still get good results, e.g. see JavaScript. But this isn't a story of success, it's a story of dynamic/politics leading to an absurd wast of human resources (dev hours) across many companies burning millions if not trillions over the years to work around language limitations. So any hypothetical language perfectly taking advantage to JIT to become "the fastest" would probably be much closer to a classical AOT language then e.g. JavaScript (as in less dynamic more strictly structured, maybe GC but maybe not or at least more restricted to reduce GC abuse possibilities, no or very limited runtime reflection, etc.). To top this of some of the benefits of JIT can be brought to AOT languages. For example you can compile a binary with LLVM in a way so that it collects statistics which then can be used to fine tune optimizations (better decide if a path is cold or which loops to unroll how much etc.). Drawback is setting it up in you deployment can add quite a bit of complexity. Another example are tools JIT some very few predetermined points, like de-visualizing/inlining virtual function calls or for one-time initialized global remove the "is initialized checks". Which means at the cost of more complex deployments doing some profiling to find some problematic vcalls and similar you can get many (but not all) the optimizations JIT languages have while having most of the benefits of AOT language. Through it only makes sense if your company/product reached a certain non-small size. Or when there is more modern tooling around to make such things easier to setup.
titzer 4y ago
A JIT with dynamic optimization for these traditionally-AOT-compiled languages has little headroom to improve the code, whereas for more dynamic languages, there's a lot of headroom. Runtime profiling has a cost, and the benefit has to outweigh that cost. Collecting receiver type profiles in order to speculatively inline has a big payoff in, e.g. Java or JavaScript, but probably not C++ or Rust, where programs have less polymorphism because programmers routinely avoid polymorphism in hot parts of the code.

I read these JIT promises decades ago, yet AOT languages like C and Rust are still the pinnacle of performance. How come?

by k__ 4y ago

I read these JIT promises decades ago, yet AOT languages like C and Rust are still the pinnacle of performance. How come?

vlovich123 4y ago

PGO can be thought of in a sense as a form of JIT (just that you have to manually capture a profile and keep it up to date to be relevant vs happening transparently at runtime). Still, it can offer benefits [1]. So clearly something that automatically JITted without the headache of managing PGO profiles could make it MUCH easier to turn globally without needing to really do a lot of work. [1] https://blog.chromium.org/2020/08/chrome-just-got-faster-with-profile.html https://blog.chromium.org/2020/08/chrome-just-got-faster-wit...

pca006132 4y ago

I think part of the reason is due to the programmer: People writing in AOT languages such as C, C++ and Rust usually care more about performance and knows more about low level details that matter for performance. For example, they may try to optimize data layout to avoid cache misses, which JavaScript programmers generally won't (if they can). Also, compiling AOT can allow programmers to look at the optimization done by the compiler and determine if they need to further tweak the code to improve performance.

dathinab 4y ago

Complexity/Time Constraints, predicability, compiler-existance and programming language design. Complexity: - JIT is only faster if it often enough recognizes the right things to JIT in the right way. - At the same time the JIT compilers share resources with the running program limiting how complex the analytics and optimizations it can do can become. This is not just a problem wrt. to hurting the main applications performance due to reducing available resources but also e.g. matters for energy usage. - Worse there is often real time time bound on how long a JIT compiler can take no matter how much resources it uses or responsiveness will suffer to much. - Similar if no additional tricks are used (which often add painful complexity) the optimizations have to be done every time the program is started again and again, hurting auto scaling or ephemeral programs. - There is also the problem that workload/data specific optimizations (which gives JIT the additional edge) rely on some data/workloads always having a specific format. If this invariants change they might become slow or might even to fallback to other code (which implies the invariant also needed to be checked). One exception is if you hint to the compiler that certain invariants (e.g. function pointers) won't change (like in the OP article). Predictability: - the performance of AOT focused language code is often more predictable making it often easier to optimize - in JIT languages you rely more on the compiler being able to detect the right patterns, this isn't always easy and even with AOT languages this sometimes isn't easy even through the compiler can do much more work to analyze the context compiler-existance: - Many AOT language can take advantage of LLVM/GCC so a lot of work has gone into optimizing them before they even started. - And while you can use LLVM to JIT it is often not viable as it's a rather large dependency with a lot of different versions which itself is optimized for throughput and not latency and is also tuned for a use-case which is allowed to take a while and consume quite a bit of resources. Language Design: - Features AOT languages are often designed with their AOT nature in mind, many JIT/Interpreted languages are not. Sometimes they outright have features which can impede JIT optimizations or at least make them more complicated/harder to find/less fast. - If you throw enough money at a badly for JIT designed language you probably still get good results, e.g. see JavaScript. But this isn't a story of success, it's a story of dynamic/politics leading to an absurd wast of human resources (dev hours) across many companies burning millions if not trillions over the years to work around language limitations. So any hypothetical language perfectly taking advantage to JIT to become "the fastest" would probably be much closer to a classical AOT language then e.g. JavaScript (as in less dynamic more strictly structured, maybe GC but maybe not or at least more restricted to reduce GC abuse possibilities, no or very limited runtime reflection, etc.). To top this of some of the benefits of JIT can be brought to AOT languages. For example you can compile a binary with LLVM in a way so that it collects statistics which then can be used to fine tune optimizations (better decide if a path is cold or which loops to unroll how much etc.). Drawback is setting it up in you deployment can add quite a bit of complexity. Another example are tools JIT some very few predetermined points, like de-visualizing/inlining virtual function calls or for one-time initialized global remove the "is initialized checks". Which means at the cost of more complex deployments doing some profiling to find some problematic vcalls and similar you can get many (but not all) the optimizations JIT languages have while having most of the benefits of AOT language. Through it only makes sense if your company/product reached a certain non-small size. Or when there is more modern tooling around to make such things easier to setup.

titzer 4y ago

A JIT with dynamic optimization for these traditionally-AOT-compiled languages has little headroom to improve the code, whereas for more dynamic languages, there's a lot of headroom. Runtime profiling has a cost, and the benefit has to outweigh that cost. Collecting receiver type profiles in order to speculatively inline has a big payoff in, e.g. Java or JavaScript, but probably not C++ or Rust, where programs have less polymorphism because programmers routinely avoid polymorphism in hot parts of the code.