3 ms·

JIT and AOT on-device is the future, I agree. How good are the JVMs at taking advantage of new instructions?

by tormeh 12y ago

bhouston 12y ago
My understanding is that JVMs do not generate any advanced instructions from standard Java code. The only time they use these instructions is if they link in hand coded C++ libraries that make use of them. Edit: Stackoverflow answer says that in rare cases the JVM can vectorize simple loops: http://stackoverflow.com/questions/10784951/do-any-jvms-jit-compilers-generate-code-that-uses-vectorized-floating-point-ins http://stackoverflow.com/questions/10784951/do-any-jvms-jit-...
- vardump 12y ago
  Just take a look at the code they generate, it's easy. In short, they do generate SIMD code, but very rarely take advantage of vectorized execution.
  - tormeh 12y ago
    Why aren't the processor manufacturers making their own JITs and AOT compilers? Surely that would be a competitive advantage.
    - vardump 12y ago
      Intel makes a famous AOT compiler, ICC. https://software.intel.com/en-us/intel-compilers https://software.intel.com/en-us/intel-compilers
      - tormeh 12y ago
        Yes, but it's for developer-compiled languages. It's not like I switch from an AMD to an Intel processor and suddenly all C code on my machine has new instructions in them. Compiled code remains the way it is. This is not what I want. What I want is either JIT or AOT-on-end-device.
        vardump 12y ago
        Binary (compiled code) JIT compilation could very well be faster than direct execution. I'm not aware of anyone currently using these techniques for running native code faster. Yet. You can also call these techniques binary translation if you like. At simplest level, one could for example dynamically replace relevant static library calls with higher performance versions. Perform function signature based optimization - when a certain known common function is found, simply replace it with a faster version. More advanced ones could for example inline functions, even those pointed by function pointers given a guard condition. It could also vectorize suitable serial code. Superoptimization techniques could also be applied to the instruction stream. http://en.wikipedia.org/wiki/Superoptimization http://en.wikipedia.org/wiki/Superoptimization Even complex optimizations are possible. Like memory access pattern translation. The JIT compiler could perform cache simulation and find a mapping between memory accesses that would increase true hardware level locality of reference. Performance could be improved by generating access transformation code when target binary tries to access memory. Like map image processing memory access code from row major to Z-order curve to increase cache hit probability. Unless memory mapped (either to other processes or physical hardware) the buffer resides the data would need to be translated back row-major only for system calls. A lazy page fault mechanism could also work in some cases.
vardump 12y ago
When I talk about JIT, I think primarily specialized form of them. I don't see much potential in JVM in its current form to be able to take advantage of this. Currently pattern detection and intrinsics ('fake' method invocation that is translated to just 1-3 instructions) are possible. JVMs should especially offer better control over data memory layout. Number of references (pointers) required needs to be radically reduced. Objects should form continuous memory regions, and have a minimal number of references to elsewhere in memory. Inlining should be used as much as practical instead of pointer chasing. Unions would help too with vectorization, but would understandably significantly complicate the optimizer. For example, an empty or a short String should be just 16 object-inlined bytes. Larger ones could still of course have references to byte or char arrays. UTF-16 should be removed in favor of UTF-8.

JIT and AOT on-device is the future, I agree. How good are the JVMs at taking advantage of new instructions?

by tormeh 12y ago

JIT and AOT on-device is the future, I agree. How good are the JVMs at taking advantage of new instructions?

My understanding is that JVMs do not generate any advanced instructions from standard Java code. The only time they use these instructions is if they link in hand coded C++ libraries that make use of them. Edit: Stackoverflow answer says that in rare cases the JVM can vectorize simple loops: http://stackoverflow.com/questions/10784951/do-any-jvms-jit-compilers-generate-code-that-uses-vectorized-floating-point-ins http://stackoverflow.com/questions/10784951/do-any-jvms-jit-...

vardump 12y ago
Just take a look at the code they generate, it's easy. In short, they do generate SIMD code, but very rarely take advantage of vectorized execution.
- tormeh 12y ago
  Why aren't the processor manufacturers making their own JITs and AOT compilers? Surely that would be a competitive advantage.
  - vardump 12y ago
    Intel makes a famous AOT compiler, ICC. https://software.intel.com/en-us/intel-compilers https://software.intel.com/en-us/intel-compilers
    - tormeh 12y ago
      Yes, but it's for developer-compiled languages. It's not like I switch from an AMD to an Intel processor and suddenly all C code on my machine has new instructions in them. Compiled code remains the way it is. This is not what I want. What I want is either JIT or AOT-on-end-device.
      - vardump 12y ago
        Binary (compiled code) JIT compilation could very well be faster than direct execution. I'm not aware of anyone currently using these techniques for running native code faster. Yet. You can also call these techniques binary translation if you like. At simplest level, one could for example dynamically replace relevant static library calls with higher performance versions. Perform function signature based optimization - when a certain known common function is found, simply replace it with a faster version. More advanced ones could for example inline functions, even those pointed by function pointers given a guard condition. It could also vectorize suitable serial code. Superoptimization techniques could also be applied to the instruction stream. http://en.wikipedia.org/wiki/Superoptimization http://en.wikipedia.org/wiki/Superoptimization Even complex optimizations are possible. Like memory access pattern translation. The JIT compiler could perform cache simulation and find a mapping between memory accesses that would increase true hardware level locality of reference. Performance could be improved by generating access transformation code when target binary tries to access memory. Like map image processing memory access code from row major to Z-order curve to increase cache hit probability. Unless memory mapped (either to other processes or physical hardware) the buffer resides the data would need to be translated back row-major only for system calls. A lazy page fault mechanism could also work in some cases.

vardump 12y ago

When I talk about JIT, I think primarily specialized form of them. I don't see much potential in JVM in its current form to be able to take advantage of this. Currently pattern detection and intrinsics ('fake' method invocation that is translated to just 1-3 instructions) are possible. JVMs should especially offer better control over data memory layout. Number of references (pointers) required needs to be radically reduced. Objects should form continuous memory regions, and have a minimal number of references to elsewhere in memory. Inlining should be used as much as practical instead of pointer chasing. Unions would help too with vectorization, but would understandably significantly complicate the optimizer. For example, an empty or a short String should be just 16 object-inlined bytes. Larger ones could still of course have references to byte or char arrays. UTF-16 should be removed in favor of UTF-8.