4 ms·

The way I sometimes think about stuff like this is that it's useful in reverse: you're going to have to write optimised code that doesn't really resemble Java.

by untog 7y ago

The way I sometimes think about stuff like this is that it's useful in reverse: you're going to have to write optimised code that doesn't really resemble Java. But if you're in a situation where there is no GPU available, that same code will execute fine on the JVM.

thu2111 7y ago
Yeah, that's TornadoVM's big thing. You just write normal code and develop it locally as you would when running on the CPU (most friendly dev environment). And then it just figures out what HW is available and uses it. This fits really nicely with scaling up on the cloud. If you have a parallel workload that you need to auto-scale very quickly, you could migrate from small CPU to big CPU, to AVX512, to GPUs to FPGAs, all automatically. Maybe it won't produce code that can beat hand-tuned Verilog but it'll do it a lot faster and cheaper. Re: no objects. That doesn't actually mean no objects. Recall this is based on Graal. Graal is really, really good at scalar replacing objects automatically, i.e. converting allocations into local variables, even when they're being e.g. allocated in a method and then returned. This is how Truffle gets such good performance: it just recursively inlines everything into one giant method and then wires up all the allocation sites to use sites. It turns out you can really use a lot of objects this way, yet at runtime they all disappear. Given it's based on the same compiler infrastructure I'd expect that to also be true of TornadoVM; perhaps a TornadoVM developer can chime in. Probably you can do method calls and working with wrapper types without hitting the limitation, or at least, it can be added without too much effort (I guess the hard part is flattening the types into arrays as value types, which may require a more sophisticated optimisation pass).
- kotselidis 7y ago
  Correct, since we employ Graal for parts of the compilation, we also enjoy the benefits of inlining and escape analysis. If an object is replaced by scalars then we can use it on a GPU. Otherwise (if a real object allocation is required) then the compiler will bail out and execution will continue normally on the CPU (with compiled code from Graal or C2).
- rbanffy 7y ago
  > when running on the CPU (most friendly dev environment) It's pretty hard to buy a computer these days that doesn't have some sort of GPU. Either it has on board graphics or it has a board connecting it to a screen.
- kllrnohj 7y ago
  > You just write normal code and develop it locally as you would when running on the CPU (most friendly dev environment). That's OpenCL's big thing. Which TornadoVM is sitting on top of. And OpenCL never really took off, it's in a bit of a weird spot currently being partially deprecated. It's a fine idea on paper, but there doesn't seem to be much of a market demand for it. Probably because it's not a thing anyone really needs. Step debugging is about the only reason to bother at all, but you really don't need any sort of heterogeneous compute migration system framework thing to pull that off. It's just a local flag or even a re-compile option. > This fits really nicely with scaling up on the cloud. If you have a parallel workload that you need to auto-scale very quickly, you could migrate from small CPU to big CPU, to AVX512, to GPUs to FPGAs, all automatically. Scaling up in the cloud is a perfect example of why TornadoVM itself isn't that interesting. Just like you can't write a normal multithreaded single process application and have it magically scale out over a network, nor can you here write a normal CPU process and have it automatically migrate between CPU & GPU. You have to design your program from the ground up around concepts like SIMT, memory coalescing, DMA bandwidth/latency, etc... These aren't just "here's some tricks to go _even faster_" things, these are "you must do this or don't even bother using the GPU at all" things. To make this work you don't need to port a different language or VM to the platform, you need the compute equivalent of Kubernetes or Hadoop or whatever other cloud scaling infrastructure system you want. Otherwise this largely boils down to a Java to OpenCL transpiler. Cool, but it's not advancing the state of art here, either.
  - jfumero 7y ago
    We are integrating TornadoVM with Apache Flink. So the idea is to execute unmodified map/reduce Flink applications automatically on heterogeneous hardware by using TornadoVM in the task-managers. This is part of a current European project (https://e2data.eu/ https://e2data.eu/).
- jfumero 7y ago
  That's correct. TornadoVM specializes the OpenCL code depending on the target device to increase performance. Also, as you pointed out, it supports some objects through the partial escape analysis by applying scalar replacements. Also, TornadoVM supports method calls and invitations to native code (e.g., Math library). The examples in the presentations are for simplicity but we have some use cases, such as KFusion Kinet with ~7k lines of Java code for computer vision (https://github.com/beehive-lab/kfusion-tornadovm https://github.com/beehive-lab/kfusion-tornadovm).

by untog 7y ago

thu2111 7y ago

Yeah, that's TornadoVM's big thing. You just write normal code and develop it locally as you would when running on the CPU (most friendly dev environment). And then it just figures out what HW is available and uses it. This fits really nicely with scaling up on the cloud. If you have a parallel workload that you need to auto-scale very quickly, you could migrate from small CPU to big CPU, to AVX512, to GPUs to FPGAs, all automatically. Maybe it won't produce code that can beat hand-tuned Verilog but it'll do it a lot faster and cheaper. Re: no objects. That doesn't actually mean no objects. Recall this is based on Graal. Graal is really, really good at scalar replacing objects automatically, i.e. converting allocations into local variables, even when they're being e.g. allocated in a method and then returned. This is how Truffle gets such good performance: it just recursively inlines everything into one giant method and then wires up all the allocation sites to use sites. It turns out you can really use a lot of objects this way, yet at runtime they all disappear. Given it's based on the same compiler infrastructure I'd expect that to also be true of TornadoVM; perhaps a TornadoVM developer can chime in. Probably you can do method calls and working with wrapper types without hitting the limitation, or at least, it can be added without too much effort (I guess the hard part is flattening the types into arrays as value types, which may require a more sophisticated optimisation pass).

kotselidis 7y ago
Correct, since we employ Graal for parts of the compilation, we also enjoy the benefits of inlining and escape analysis. If an object is replaced by scalars then we can use it on a GPU. Otherwise (if a real object allocation is required) then the compiler will bail out and execution will continue normally on the CPU (with compiled code from Graal or C2).
rbanffy 7y ago
> when running on the CPU (most friendly dev environment) It's pretty hard to buy a computer these days that doesn't have some sort of GPU. Either it has on board graphics or it has a board connecting it to a screen.
kllrnohj 7y ago
> You just write normal code and develop it locally as you would when running on the CPU (most friendly dev environment). That's OpenCL's big thing. Which TornadoVM is sitting on top of. And OpenCL never really took off, it's in a bit of a weird spot currently being partially deprecated. It's a fine idea on paper, but there doesn't seem to be much of a market demand for it. Probably because it's not a thing anyone really needs. Step debugging is about the only reason to bother at all, but you really don't need any sort of heterogeneous compute migration system framework thing to pull that off. It's just a local flag or even a re-compile option. > This fits really nicely with scaling up on the cloud. If you have a parallel workload that you need to auto-scale very quickly, you could migrate from small CPU to big CPU, to AVX512, to GPUs to FPGAs, all automatically. Scaling up in the cloud is a perfect example of why TornadoVM itself isn't that interesting. Just like you can't write a normal multithreaded single process application and have it magically scale out over a network, nor can you here write a normal CPU process and have it automatically migrate between CPU & GPU. You have to design your program from the ground up around concepts like SIMT, memory coalescing, DMA bandwidth/latency, etc... These aren't just "here's some tricks to go _even faster_" things, these are "you must do this or don't even bother using the GPU at all" things. To make this work you don't need to port a different language or VM to the platform, you need the compute equivalent of Kubernetes or Hadoop or whatever other cloud scaling infrastructure system you want. Otherwise this largely boils down to a Java to OpenCL transpiler. Cool, but it's not advancing the state of art here, either.
- jfumero 7y ago
  We are integrating TornadoVM with Apache Flink. So the idea is to execute unmodified map/reduce Flink applications automatically on heterogeneous hardware by using TornadoVM in the task-managers. This is part of a current European project (https://e2data.eu/ https://e2data.eu/).
jfumero 7y ago
That's correct. TornadoVM specializes the OpenCL code depending on the target device to increase performance. Also, as you pointed out, it supports some objects through the partial escape analysis by applying scalar replacements. Also, TornadoVM supports method calls and invitations to native code (e.g., Math library). The examples in the presentations are for simplicity but we have some use cases, such as KFusion Kinet with ~7k lines of Java code for computer vision (https://github.com/beehive-lab/kfusion-tornadovm https://github.com/beehive-lab/kfusion-tornadovm).