4 ms·

TornadoVM v1.0 Released

40 points by lichtenberger 3y ago

lichtenberger 3y ago
Here's the project website: https://www.tornadovm.org/ https://www.tornadovm.org/
MrBuddyCasino 3y ago
I wondered about the performance delta to "native" CUDA kernels, and was disappointed there are only benchmark comparisons to single-threaded Java on their website.
- ckotseli 3y ago
  There are some comparisons in some of our group's papers (e.g. [1] Figure 9 where we compare against optimized OpenCL kernels). In general, from our experience, TornadoVM's relative performance to optimized CUDA or OpenCL kernels depends on the kind of the kernel. Our data suggests that it can perform between 40% and 90% of the raw execution time of native kernels. With TornadoVM v1.0 and the integration with native memory via Project Panama, such performance comparisons would be more straightforward since the memory subsystem of TornadoVM can be further optimized. Of course with this performance, someone would also benefit from ease in programmability since the starting point is existing or new Java code that can be written as single-threaded. For more power users TornadoVM offers also the Kernel API which is very comparable to OpenCL that allow users to extract more performance by using optimizations such as grid scheduling, local memory, etc.: https://tornadovm.readthedocs.io/en/latest/programming.html#kernel-api https://tornadovm.readthedocs.io/en/latest/programming.html#... [1] https://research.manchester.ac.uk/files/190177400/MPAPADIMITRIOU_VEE2021_GPU_MEMORY_JIT_Preprint.pdf https://research.manchester.ac.uk/files/190177400/MPAPADIMIT...
  - MrBuddyCasino 3y ago
    That’s actually pretty decent, thanks for the detailed answer.
    - jfumero 3y ago
      It seems the Manchester website is down. Link to the paper: https://github.com/jjfumero/jjfumero.github.io/blob/master/files/papers/2021/vee21-GPUMemoryJIT.pdf https://github.com/jjfumero/jjfumero.github.io/blob/master/f... Just to complement the previous answer. It varies from kernel to kernel. Regarding reductions, the strategy implemented in the TornadoVM JIT compiler allows us to execute reductions offloaded from Java sequential code within 85% of the hand-written OpenCL Kernels: https://www.researchgate.net/publication/327871451_Using_Compiler_Snippets_to_Exploit_Parallelism_on_Heterogeneous_Hardware_A_Java_Reduction_Case_Study https://www.researchgate.net/publication/327871451_Using_Com...
JonChesterfield 3y ago
The main pages suggests you can take java code and it runs faster if there are gpus present. I like implicit concurrency so looked further. The documentation on using it suggest there's a parallel annotation which tries to build something, while the main API is at https://tornadovm.readthedocs.io/en/latest/programming.html#kernel-api https://tornadovm.readthedocs.io/en/latest/programming.html#.... That's the traditional clunky kernel interface from opencl et al. So I think this is roughly exposing the usual GPU programming model, but in Java instead of C++, with some early steps towards openmp/openmp c style concurrency available as well. Interesting project but not what the marketing page got my hopes up for.
- ckotseli 3y ago
  TornadoVM has two distinct APIs: 1) Loop-parallel API, and 2) Kernel API. We can use one or the other or both together. The loop parallel API requires only one annotation to be placed to the induction variable of a normal Java loop. The kernel API is the more heavy-handed API for more power users which, as you said, follows similar style to OpenCL and exposes more hardware-specific programming elements to the user.

40 points by lichtenberger 3y ago

Here's the project website: https://www.tornadovm.org/ https://www.tornadovm.org/

I wondered about the performance delta to "native" CUDA kernels, and was disappointed there are only benchmark comparisons to single-threaded Java on their website.

ckotseli 3y ago
There are some comparisons in some of our group's papers (e.g. [1] Figure 9 where we compare against optimized OpenCL kernels). In general, from our experience, TornadoVM's relative performance to optimized CUDA or OpenCL kernels depends on the kind of the kernel. Our data suggests that it can perform between 40% and 90% of the raw execution time of native kernels. With TornadoVM v1.0 and the integration with native memory via Project Panama, such performance comparisons would be more straightforward since the memory subsystem of TornadoVM can be further optimized. Of course with this performance, someone would also benefit from ease in programmability since the starting point is existing or new Java code that can be written as single-threaded. For more power users TornadoVM offers also the Kernel API which is very comparable to OpenCL that allow users to extract more performance by using optimizations such as grid scheduling, local memory, etc.: https://tornadovm.readthedocs.io/en/latest/programming.html#kernel-api https://tornadovm.readthedocs.io/en/latest/programming.html#... [1] https://research.manchester.ac.uk/files/190177400/MPAPADIMITRIOU_VEE2021_GPU_MEMORY_JIT_Preprint.pdf https://research.manchester.ac.uk/files/190177400/MPAPADIMIT...
- MrBuddyCasino 3y ago
  That’s actually pretty decent, thanks for the detailed answer.
  - jfumero 3y ago
    It seems the Manchester website is down. Link to the paper: https://github.com/jjfumero/jjfumero.github.io/blob/master/files/papers/2021/vee21-GPUMemoryJIT.pdf https://github.com/jjfumero/jjfumero.github.io/blob/master/f... Just to complement the previous answer. It varies from kernel to kernel. Regarding reductions, the strategy implemented in the TornadoVM JIT compiler allows us to execute reductions offloaded from Java sequential code within 85% of the hand-written OpenCL Kernels: https://www.researchgate.net/publication/327871451_Using_Compiler_Snippets_to_Exploit_Parallelism_on_Heterogeneous_Hardware_A_Java_Reduction_Case_Study https://www.researchgate.net/publication/327871451_Using_Com...

JonChesterfield 3y ago

The main pages suggests you can take java code and it runs faster if there are gpus present. I like implicit concurrency so looked further. The documentation on using it suggest there's a parallel annotation which tries to build something, while the main API is at https://tornadovm.readthedocs.io/en/latest/programming.html#kernel-api https://tornadovm.readthedocs.io/en/latest/programming.html#.... That's the traditional clunky kernel interface from opencl et al. So I think this is roughly exposing the usual GPU programming model, but in Java instead of C++, with some early steps towards openmp/openmp c style concurrency available as well. Interesting project but not what the marketing page got my hopes up for.

ckotseli 3y ago
TornadoVM has two distinct APIs: 1) Loop-parallel API, and 2) Kernel API. We can use one or the other or both together. The loop parallel API requires only one annotation to be placed to the induction variable of a normal Java loop. The kernel API is the more heavy-handed API for more power users which, as you said, follows similar style to OpenCL and exposes more hardware-specific programming elements to the user.