4 ms·

Can you explain why it's not technically possible. JVM has had decades of experience at optimally translating bytecode to machine code and can take advantage o

Can you explain why it's not technically possible. JVM has had decades of experience at optimally translating bytecode to machine code and can take advantage of SIMD, AVX etc when needed. Most hand-written C code is far from optimal.

10000truths 3y ago
C compilers also have decades of experience optimally translating C code into machine code, and they are arguably more capable of emitting SIMD (good luck trying to use cutting edge AVX-512 intrinsics like vpopcntdq with the JVM). The fact is that there is nothing a JIT compiler can do that an AOT compiler can't do, but in the case of AOT, the resources spent compiling the code are amortized to effectively 0, whereas that resource cost is borne upon every program startup for a JIT engine.
- pjmlp 3y ago
  C compilers only have one opportunity to do that once, at compile time, if the developer was lucky with their data set used to train the PGO output, maybe the outcome is greatly improved. Modern JVMs, not only have the JIT being able to use actual production data, they are able to cache PGO data between execution runs, and reach an optimimal set of heuristics throughout execution time. And on Android, those PGO files are even shared between devices via Play Store.
- neonsunset 3y ago
  That's not necessarily true, on JIT vs AOT split. I'm mostly going off of how the divergence in available optimization is starting to look like in .NET after introduction of Native AOT with light research into LLVM and various optimization-adjacent Rust crates. In particular, with JIT, you are able to initialize certain readonly data once, and then, on recompilation to a more optimized version, bake such data as JIT constants right into emitted machine code. This is not possible with AOT. Same applies for all kinds of in-runtime profiling/analysis and recompilation to incorporate a collected profile according to this exact run of an application. JIT also offers the ability to load modules dynamically in the form of bytecode without having to have a strict machine-level ABI, only the bytecode one, which allows for efficient generics that cross modules, as well as cross-module function inlining. And last but not least - there is no need to pick the least common denominator in supported hardware features as the code can be compiled to use the latest features provided by hardware like AVX512. On the other hand, pure AOT means a frozen world which allows the compiler to know exact types and paths the code can take, performing exact devirtualization and much more aggressive preinitialization on code that accepts constant data. It also means bigger leeway in the time the compiler can spend on optimizing code. Historically, GCC and LLVM have been more advanced than their JIT counterparts because of different tradeoffs more favouring to absolute performance of the emitted code as well as simply higher amount of man hours invested in developing them (e.g. .NET punches above it's weight class despite being worked on by a smaller team vs OpenJDK or LLVM).
dzaima 3y ago
A couple weeks ago I managed to get a nice setup for viewing Java JIT disassembly[1], and did a bunch of ad-hoc tests on OpenJDK 21 (and some on whatever build of 23 was on openjdk.org). do manage to vectorize a decent amount of vectorizable loops, but semi-often missed some improvements, and some trivial things didn't vectorize at all (most amusingly, a loop summing an int[] to an int didn't get vectorized, while int[] to long is). Scalar code codegen is also quite mediocre. GCC & clang, in comparison, often produce assembly which I'd consider optimal, or close to it, for vectorizable loops, and are noticably better than OpenJDK for scalar code. [1]: it's a bit messy to use but: https://github.com/dzaima/grr#java-jit https://github.com/dzaima/grr#java-jit
- saagarjha 3y ago
  GCC and clang produce far from optimal code when vectorizing. Anyone doing serious work is unlikely to rely on the autovectorizer without consulting the output religiously.
  - dzaima 3y ago
    There are certainly a lot of things they may do suboptimally (in my experience: when there's register pressure; things that aren't nicely elementwise (bit arrays); preparation & tail handling can get pretty excessive; basically no tuning for making better use of ports) and of course you should always be checking the output (this is true for any optimization effort, vectorized or not), but for most things IME they do just fine (assuming the thing in question is vectorized at all). At the very least it's still miles above OpenJDK.
    - saagarjha 3y ago
      I've only really seen decent results on the simplest of examples, like a bounded loop with a basic operation in it (like, adding all the numbers in an array). As soon as you go beyond that I see much worse code: reloading things inside the loop body that really should have been hoisted out, spilling for no good reason, weird contortions to shove certain things into scalar registers in the middle only to take them out again. And, of course, if your control flow is not trivial the compiler will often not recognize the loop as vectorizable at all.

10000truths 3y ago

C compilers also have decades of experience optimally translating C code into machine code, and they are arguably more capable of emitting SIMD (good luck trying to use cutting edge AVX-512 intrinsics like vpopcntdq with the JVM). The fact is that there is nothing a JIT compiler can do that an AOT compiler can't do, but in the case of AOT, the resources spent compiling the code are amortized to effectively 0, whereas that resource cost is borne upon every program startup for a JIT engine.

pjmlp 3y ago
C compilers only have one opportunity to do that once, at compile time, if the developer was lucky with their data set used to train the PGO output, maybe the outcome is greatly improved. Modern JVMs, not only have the JIT being able to use actual production data, they are able to cache PGO data between execution runs, and reach an optimimal set of heuristics throughout execution time. And on Android, those PGO files are even shared between devices via Play Store.
neonsunset 3y ago
That's not necessarily true, on JIT vs AOT split. I'm mostly going off of how the divergence in available optimization is starting to look like in .NET after introduction of Native AOT with light research into LLVM and various optimization-adjacent Rust crates. In particular, with JIT, you are able to initialize certain readonly data once, and then, on recompilation to a more optimized version, bake such data as JIT constants right into emitted machine code. This is not possible with AOT. Same applies for all kinds of in-runtime profiling/analysis and recompilation to incorporate a collected profile according to this exact run of an application. JIT also offers the ability to load modules dynamically in the form of bytecode without having to have a strict machine-level ABI, only the bytecode one, which allows for efficient generics that cross modules, as well as cross-module function inlining. And last but not least - there is no need to pick the least common denominator in supported hardware features as the code can be compiled to use the latest features provided by hardware like AVX512. On the other hand, pure AOT means a frozen world which allows the compiler to know exact types and paths the code can take, performing exact devirtualization and much more aggressive preinitialization on code that accepts constant data. It also means bigger leeway in the time the compiler can spend on optimizing code. Historically, GCC and LLVM have been more advanced than their JIT counterparts because of different tradeoffs more favouring to absolute performance of the emitted code as well as simply higher amount of man hours invested in developing them (e.g. .NET punches above it's weight class despite being worked on by a smaller team vs OpenJDK or LLVM).

dzaima 3y ago

A couple weeks ago I managed to get a nice setup for viewing Java JIT disassembly[1], and did a bunch of ad-hoc tests on OpenJDK 21 (and some on whatever build of 23 was on openjdk.org). do manage to vectorize a decent amount of vectorizable loops, but semi-often missed some improvements, and some trivial things didn't vectorize at all (most amusingly, a loop summing an int[] to an int didn't get vectorized, while int[] to long is). Scalar code codegen is also quite mediocre. GCC & clang, in comparison, often produce assembly which I'd consider optimal, or close to it, for vectorizable loops, and are noticably better than OpenJDK for scalar code. [1]: it's a bit messy to use but: https://github.com/dzaima/grr#java-jit https://github.com/dzaima/grr#java-jit

saagarjha 3y ago
GCC and clang produce far from optimal code when vectorizing. Anyone doing serious work is unlikely to rely on the autovectorizer without consulting the output religiously.
- dzaima 3y ago
  There are certainly a lot of things they may do suboptimally (in my experience: when there's register pressure; things that aren't nicely elementwise (bit arrays); preparation & tail handling can get pretty excessive; basically no tuning for making better use of ports) and of course you should always be checking the output (this is true for any optimization effort, vectorized or not), but for most things IME they do just fine (assuming the thing in question is vectorized at all). At the very least it's still miles above OpenJDK.
  - saagarjha 3y ago
    I've only really seen decent results on the simplest of examples, like a bounded loop with a basic operation in it (like, adding all the numbers in an array). As soon as you go beyond that I see much worse code: reloading things inside the loop body that really should have been hoisted out, spilling for no good reason, weird contortions to shove certain things into scalar registers in the middle only to take them out again. And, of course, if your control flow is not trivial the compiler will often not recognize the loop as vectorizable at all.