7 ms·

The theoretic fundamental benefit is you are able to optimize for code paths practically occurring during the runtime with particular set of data or inputs. The

by varjag 8y ago

The theoretic fundamental benefit is you are able to optimize for code paths practically occurring during the runtime with particular set of data or inputs. There is no way to know that at compile time, hence it's often touted as a way to make things "faster than C" with sufficiently smart JIT compiler. But of course no JIT compiler ever is sufficiently smart and any execution path benefit gained is lost to poor cache discipline of VMs. And the whole JIT/hotspot concept was really a hindsight attempt to get away from abysmal performance of VM based languages.

hyperman1 8y ago
There is another very practical benefit: You can use all features available on your PC. E.g. when you compile an application C-style, you set the target architecture to the lowest CPU you want to support. If this is , say, SSE, then no SSE2 instructions will be used by the program and the sometimes huge performance gains from them are lost. A VM like hotspot can generate code for this specific machine. If you have SSE2, it will generate SSE2 instructions. If you dont, it wont. Recently, I see more applications choosing code paths based on CPUID, but this only works for CPUs existing at compile time. If you app was compiled in 2016, in wont use CPU features from 2018. If your java app from 2016 runs on a JRE from 2018, it will use them.
- cm2187 8y ago
  But don't static compilers generate checks for the availability of these features and ship both versions? I guess it is rather to take advantage of new features that were not available when the code was compiled.
- varjag 8y ago
  I would really like to see a Java program beating a C program in this use case. Say, outperforming it (or even matching to same order of magnitude) on DCT/IDCT even by using SSE2 vs SSE/MMX. Can only say am immensely sceptical. Another thing is these kind of gains with instruction level efficiencies are exceedingly rare and your typical VM program never runs any code where SIMD extensions would make a difference.
  - deleted 8y ago
    [deleted]
  - hyperman1 8y ago
    This article is not what you ask, but it is an example of a VM beating native: https://nullprogram.com/blog/2018/05/27/ https://nullprogram.com/blog/2018/05/27/ Java uses the same technique. Basically, a JIT allows you to skip the overhead of the GOT. Devirtualization is another case. I have personally seen Java beating an existing C++ program for the reason I describe, but I cant publish the results here. You have to program carefully or you loose all speed benefits because of pointer chasing. But it is possible.
    - hyperman1 8y ago
      I'll add another one: Inlining across libraries. AFAIK, Even if you statically precompile bytecode on the target system a la dotnet this is not possible as the individual library and executable files are barriers the compiler can't cross.
    - wolfgke 8y ago
      > Basically, a JIT allows you to skip the overhead of the GOT. This is a problem that is GNU/Linux-specific. Windows DLLs are not PIC (position-independent code), so they do not suffer from this problem and don't need a GOT. See > https://www.symantec.com/connect/articles/dynamic-linking-linux-and-windows-part-one https://www.symantec.com/connect/articles/dynamic-linking-li... for details. The method that Windows uses is much faster in the average case, but slower in the "bad" case. So GNU/Linux vs Windows have different performance tradeoffs.
      - hyperman1 8y ago
        Yes and no: there is another indirection mechanism at work on windows: All results for imports in the import table are at the beginning of the DLL's data segment. So a call to another DLL first has to fetch the data from there, then do an indirect call. In x86 assembler: some_import_address dd some_import ; will be filled in by dll import [... lots of unrelated code and data...] call [some_import_address] a JIT can read the address from code and do directly: call some_import that's one less memory load. The benefit is that the dynamic linking does not modify the code segment, so there are less private, modified pages mmapped, thus more sharing and less memory usage.
- de_watcher 8y ago
  In other words "an obfuscated Gentoo".
pron 8y ago
You're forgetting that the whole idea is not to compile the same language better, but to compile different languages (that are supposed to offer better productivity) to have similar performance to C. Obviously, languages such as Javascript or Ruby cannot possibly be compiled AOT with good performance, but even Java relies heavily on virtual dispatch, and a JIT can inline through virtual calls. Languages that don't rely on virtual dispatch as much will benefit less from a JIT. Today, as we have both AOT and JIT compilers for Java, we can see that the JIT does, indeed, generate better code. By how much depends on your specific use-case. > lost to poor cache discipline of VMs What are you referring to? If you're talking about memory layout, then this has nothing to do with the VM and everything to do with the language design. The reason Java has little control over memory layout was that it didn't make much of a difference in the '90s; now it does, so Java is getting control of layout.
- cm2187 8y ago
  But you don't need JIT for that. You need an intermediary language (ByteCode / MSIL). In a non-JIT world you would use ByteCode/MSIL libraries, and just before deploying, statically compile them for the platform they will be deployed to. JIT just means you are doing that step everytime the program runs (+/- caching) for every machine.
  - pron 8y ago
    You cannot efficiently compile ahead of time languages whose semantics heavily rely on dynamic dispatch. AOT profile-guided optimization would help, but it carries added effort (and still cannot be as aggressive if there is no possibility of deoptimization and recompilation by a JIT). Intermediate representation is completely irrelevant. Java now offers "deploy-time" AOT compilation, and the results are not as good as a JIT (YMMV).
    - pjmlp 8y ago
      Java now offers "deploy-time" AOT compilation for free. It has been available since the early days in commercial JDKs and initially it was the deployment option on IBM mainframes, by converting JVM bytecodes into the bytecodes of the mainframe execution environments.
nickpsecurity 8y ago
"There is no way to know that at compile time" Profile-guided, whole-program optimization can do that for whatever execution patterns the developer feeds them. It will be faster than the JIT's because it has more time to analyze and optimize. The JIT will be better on the exceptional cases the AOT compiler didn't optimize for. In many apps, I'd much rather have the AOT approach than the JIT approach. Heck, some routines in fast path might even get superoptimized if developer wants. :)

by varjag 8y ago

hyperman1 8y ago

There is another very practical benefit: You can use all features available on your PC. E.g. when you compile an application C-style, you set the target architecture to the lowest CPU you want to support. If this is , say, SSE, then no SSE2 instructions will be used by the program and the sometimes huge performance gains from them are lost. A VM like hotspot can generate code for this specific machine. If you have SSE2, it will generate SSE2 instructions. If you dont, it wont. Recently, I see more applications choosing code paths based on CPUID, but this only works for CPUs existing at compile time. If you app was compiled in 2016, in wont use CPU features from 2018. If your java app from 2016 runs on a JRE from 2018, it will use them.

cm2187 8y ago
But don't static compilers generate checks for the availability of these features and ship both versions? I guess it is rather to take advantage of new features that were not available when the code was compiled.
varjag 8y ago
I would really like to see a Java program beating a C program in this use case. Say, outperforming it (or even matching to same order of magnitude) on DCT/IDCT even by using SSE2 vs SSE/MMX. Can only say am immensely sceptical. Another thing is these kind of gains with instruction level efficiencies are exceedingly rare and your typical VM program never runs any code where SIMD extensions would make a difference.
- deleted 8y ago
  [deleted]
- hyperman1 8y ago
  This article is not what you ask, but it is an example of a VM beating native: https://nullprogram.com/blog/2018/05/27/ https://nullprogram.com/blog/2018/05/27/ Java uses the same technique. Basically, a JIT allows you to skip the overhead of the GOT. Devirtualization is another case. I have personally seen Java beating an existing C++ program for the reason I describe, but I cant publish the results here. You have to program carefully or you loose all speed benefits because of pointer chasing. But it is possible.
  - hyperman1 8y ago
    I'll add another one: Inlining across libraries. AFAIK, Even if you statically precompile bytecode on the target system a la dotnet this is not possible as the individual library and executable files are barriers the compiler can't cross.
  - wolfgke 8y ago
    > Basically, a JIT allows you to skip the overhead of the GOT. This is a problem that is GNU/Linux-specific. Windows DLLs are not PIC (position-independent code), so they do not suffer from this problem and don't need a GOT. See > https://www.symantec.com/connect/articles/dynamic-linking-linux-and-windows-part-one https://www.symantec.com/connect/articles/dynamic-linking-li... for details. The method that Windows uses is much faster in the average case, but slower in the "bad" case. So GNU/Linux vs Windows have different performance tradeoffs.
    - hyperman1 8y ago
      Yes and no: there is another indirection mechanism at work on windows: All results for imports in the import table are at the beginning of the DLL's data segment. So a call to another DLL first has to fetch the data from there, then do an indirect call. In x86 assembler: some_import_address dd some_import ; will be filled in by dll import [... lots of unrelated code and data...] call [some_import_address] a JIT can read the address from code and do directly: call some_import that's one less memory load. The benefit is that the dynamic linking does not modify the code segment, so there are less private, modified pages mmapped, thus more sharing and less memory usage.
de_watcher 8y ago
In other words "an obfuscated Gentoo".

pron 8y ago

You're forgetting that the whole idea is not to compile the same language better, but to compile different languages (that are supposed to offer better productivity) to have similar performance to C. Obviously, languages such as Javascript or Ruby cannot possibly be compiled AOT with good performance, but even Java relies heavily on virtual dispatch, and a JIT can inline through virtual calls. Languages that don't rely on virtual dispatch as much will benefit less from a JIT. Today, as we have both AOT and JIT compilers for Java, we can see that the JIT does, indeed, generate better code. By how much depends on your specific use-case. > lost to poor cache discipline of VMs What are you referring to? If you're talking about memory layout, then this has nothing to do with the VM and everything to do with the language design. The reason Java has little control over memory layout was that it didn't make much of a difference in the '90s; now it does, so Java is getting control of layout.

cm2187 8y ago
But you don't need JIT for that. You need an intermediary language (ByteCode / MSIL). In a non-JIT world you would use ByteCode/MSIL libraries, and just before deploying, statically compile them for the platform they will be deployed to. JIT just means you are doing that step everytime the program runs (+/- caching) for every machine.
- pron 8y ago
  You cannot efficiently compile ahead of time languages whose semantics heavily rely on dynamic dispatch. AOT profile-guided optimization would help, but it carries added effort (and still cannot be as aggressive if there is no possibility of deoptimization and recompilation by a JIT). Intermediate representation is completely irrelevant. Java now offers "deploy-time" AOT compilation, and the results are not as good as a JIT (YMMV).
  - pjmlp 8y ago
    Java now offers "deploy-time" AOT compilation for free. It has been available since the early days in commercial JDKs and initially it was the deployment option on IBM mainframes, by converting JVM bytecodes into the bytecodes of the mainframe execution environments.

nickpsecurity 8y ago

"There is no way to know that at compile time" Profile-guided, whole-program optimization can do that for whatever execution patterns the developer feeds them. It will be faster than the JIT's because it has more time to analyze and optimize. The JIT will be better on the exceptional cases the AOT compiler didn't optimize for. In many apps, I'd much rather have the AOT approach than the JIT approach. Heck, some routines in fast path might even get superoptimized if developer wants. :)