13 ms·

Tl;dr: JIT is a compiler that optimuzes certain parts of code after interpreter see it as hot (e.g. was executed many time). Thank to runtime information it can

Tl;dr: JIT is a compiler that optimuzes certain parts of code after interpreter see it as hot (e.g. was executed many time). Thank to runtime information it can occasionally exceed performance of statically compiled language.

kipply 6y ago
except Julia because Julia <3
- dunefox 6y ago
  What do you mean?
  - kipply 6y ago
    Julia compiles at runtime which makes it a JIT but it does not actually use runtime information to compile (except to know what to compile and maybe some other things) or have a concept of "hotness" (this is in the post~)
- MaxBarraclough 6y ago
  Or .Net, which entirely lacks an interpreter, and JIT compiles everything that it executes. (Like Julia, if I understand correctly.) JIT != mixed-mode execution.
  - saagarjha 6y ago
    Mono has one, AFAIK: https://www.mono-project.com/news/2017/11/13/mono-interpreter/ https://www.mono-project.com/news/2017/11/13/mono-interprete...
  - chrisseaton 6y ago
    Some people make a distinction between 'just-in-time compilation' (like .NET - it's normal static compilation, but done just before first execution) and 'dynamic compilation' (like Java, it choses when to compile, uses dynamic information, may re-compile, etc.)
    - ddragon 6y ago
      In the Julia community they frequently call it Just Ahead-Of-Time the compiler strategy of making the inference of all downstream types and methods to dispatch and compiling all at once. And the period while purely running a static program "world age", which usually lasts until you go back to the global namespace or you need any dynamic feature. Julia does not have an interpreter outside of the debugger.
dialamac 6y ago
> Thank to runtime information it can occasionally exceed performance of statically compiled language. Interestingly in 30 years I have not once heard of a case where this theoretical benefit has manifested as a clear advantage in any real world application when looking at the system as a whole... amdahls law and all that. You can always hand tune the 1-10% hotspots for reasonable cost most of the time, and even static tools can do PGO which generally gets you where JIT would anyway.
- chrisseaton 6y ago
  > in 30 years I have not once heard of a case where this theoretical benefit has manifested as a clear advantage in any real world application Today you make make a very direct empirical comparison to see this - using the Graal compiler. This lets you compile exactly the same Java code either ahead-of-time or just-in-time, but using the same compiler logic except for the runtime information available when running just-in-time. The just-in-time code is (ignoring startup and warmup time) in my experience always faster, due to the extra runtime information.
  - darksaints 6y ago
    Have you tried graal's PGO?
    - chrisseaton 6y ago
      No unfortunately - it's a closed-source Enterprise feature I believe. But the current differential is pretty large and nobody is shouting loud that they can fix it using the PGO that I've heard. And what will the PGO determine that a JIT can't also do?
      - darksaints 6y ago
        A number of things actually. For one, most JIT implementations only optimize once, instead of continuously. The result is machine code that is optimized for the sorts of things done at startup, as opposed to steady state operation. For example, I have a Play app that takes a minute to start up. The JIT does a great job of optimizing the code that is called during the setup process, but the API code itself doesn't get much optimization. With PGO, I can get a more representative profiling dataset, allowing the JIT to see actual production loads instead of startup loads. And for CLI apps, PGO code starts up fast and never slows down to profile or optimize.
glitchc 6y ago
I don’t believe JITs can ever beat AOT compilation. However, if it can achieve parity in the execution of the critical path, that is sufficient to call it a win.
- Reelin 6y ago
  > I don’t believe JITs can ever beat AOT compilation. In certain cases [1] JIT can beat AOT by a decent margin because it has access to live information that isn't available at compile time. In theory you could AOT compile, profile, and recompile but in practice my understanding is that nothing beats doing it live. Consider that if the workload suddenly changes in production and your heuristic violates an invariant as a result, you can deoptimize, reprofile, and recompile everything on the fly. But wait, you say! Can't an AOT compiler do that? But no, the state space is almost certainly going to be too large by many orders of magnitude. So unless your AOT compiler effectively inserts a JIT compiler ... [1] Sorry, no examples to hand. Nobody knows but I'm actually a dog. Don't trust me!
  - caballeto 6y ago
    Dart VM is a good example. Their JIT is frequently faster than the AOT compiled code, although they may improve the AOT speed in future releases.
- pjmlp 6y ago
  Yes they can in these scenarios: - Dynamic method calls that get de-virtualized, because there is only one visible implementation, including across shared libraries - Inlining across shared libraries - Adaptation of new CPU opcodes, for example a Java application written in 2000 could in theory make use of AVX, without being recompiled. - In the same vein, that is how shading languages work

kipply 6y ago

except Julia because Julia <3

dunefox 6y ago
What do you mean?
- kipply 6y ago
  Julia compiles at runtime which makes it a JIT but it does not actually use runtime information to compile (except to know what to compile and maybe some other things) or have a concept of "hotness" (this is in the post~)
MaxBarraclough 6y ago
Or .Net, which entirely lacks an interpreter, and JIT compiles everything that it executes. (Like Julia, if I understand correctly.) JIT != mixed-mode execution.
- saagarjha 6y ago
  Mono has one, AFAIK: https://www.mono-project.com/news/2017/11/13/mono-interpreter/ https://www.mono-project.com/news/2017/11/13/mono-interprete...
- chrisseaton 6y ago
  Some people make a distinction between 'just-in-time compilation' (like .NET - it's normal static compilation, but done just before first execution) and 'dynamic compilation' (like Java, it choses when to compile, uses dynamic information, may re-compile, etc.)
  - ddragon 6y ago
    In the Julia community they frequently call it Just Ahead-Of-Time the compiler strategy of making the inference of all downstream types and methods to dispatch and compiling all at once. And the period while purely running a static program "world age", which usually lasts until you go back to the global namespace or you need any dynamic feature. Julia does not have an interpreter outside of the debugger.

dialamac 6y ago

> Thank to runtime information it can occasionally exceed performance of statically compiled language. Interestingly in 30 years I have not once heard of a case where this theoretical benefit has manifested as a clear advantage in any real world application when looking at the system as a whole... amdahls law and all that. You can always hand tune the 1-10% hotspots for reasonable cost most of the time, and even static tools can do PGO which generally gets you where JIT would anyway.

chrisseaton 6y ago
> in 30 years I have not once heard of a case where this theoretical benefit has manifested as a clear advantage in any real world application Today you make make a very direct empirical comparison to see this - using the Graal compiler. This lets you compile exactly the same Java code either ahead-of-time or just-in-time, but using the same compiler logic except for the runtime information available when running just-in-time. The just-in-time code is (ignoring startup and warmup time) in my experience always faster, due to the extra runtime information.
- darksaints 6y ago
  Have you tried graal's PGO?
  - chrisseaton 6y ago
    No unfortunately - it's a closed-source Enterprise feature I believe. But the current differential is pretty large and nobody is shouting loud that they can fix it using the PGO that I've heard. And what will the PGO determine that a JIT can't also do?
    - darksaints 6y ago
      A number of things actually. For one, most JIT implementations only optimize once, instead of continuously. The result is machine code that is optimized for the sorts of things done at startup, as opposed to steady state operation. For example, I have a Play app that takes a minute to start up. The JIT does a great job of optimizing the code that is called during the setup process, but the API code itself doesn't get much optimization. With PGO, I can get a more representative profiling dataset, allowing the JIT to see actual production loads instead of startup loads. And for CLI apps, PGO code starts up fast and never slows down to profile or optimize.

glitchc 6y ago

I don’t believe JITs can ever beat AOT compilation. However, if it can achieve parity in the execution of the critical path, that is sufficient to call it a win.

Reelin 6y ago
> I don’t believe JITs can ever beat AOT compilation. In certain cases [1] JIT can beat AOT by a decent margin because it has access to live information that isn't available at compile time. In theory you could AOT compile, profile, and recompile but in practice my understanding is that nothing beats doing it live. Consider that if the workload suddenly changes in production and your heuristic violates an invariant as a result, you can deoptimize, reprofile, and recompile everything on the fly. But wait, you say! Can't an AOT compiler do that? But no, the state space is almost certainly going to be too large by many orders of magnitude. So unless your AOT compiler effectively inserts a JIT compiler ... [1] Sorry, no examples to hand. Nobody knows but I'm actually a dog. Don't trust me!
- caballeto 6y ago
  Dart VM is a good example. Their JIT is frequently faster than the AOT compiled code, although they may improve the AOT speed in future releases.
pjmlp 6y ago
Yes they can in these scenarios: - Dynamic method calls that get de-virtualized, because there is only one visible implementation, including across shared libraries - Inlining across shared libraries - Adaptation of new CPU opcodes, for example a Java application written in 2000 could in theory make use of AVX, without being recompiled. - In the same vein, that is how shading languages work