4 ms·

>And when you think about how a GPU works, it completely makes sense. A high level language for a GPU will not look like Java Can someone explain this further?

>And when you think about how a GPU works, it completely makes sense. A high level language for a GPU will not look like Java Can someone explain this further? I am genuinely interested in learning what makes Java etc not suitable for GPUs.

AnimalMuppet 7y ago
Other way around - GPUs are not suitable for Java. Java is a dynamically-allocated, garbage-collected language. So far as I know, GPU code doesn't have the ability to dynamically allocate memory. (I mean, I guess you could, but you'd probably need one memory pool per parallel computational unit. Otherwise, you'd need a global lock on the global memory pool.) Java is a general programming language. GPUs are not general-purpose processors. That's off the top of my head. There may be other reasons as well.
- pjmlp 7y ago
  CUDA can dynamically allocate, more recente versions even support unified shared memory blocks with the host processor. It is also possible to do native memory allocation in Java. Imposing restrictions on a Java subset is hardly any different of imposing restrictions on C and C++, which apparently is ok to do.
  - whizzter 7y ago
    The big difference that makes Java cumbersome is that you always have object references (ie pointers), thus an array of objects is really an array of pointers to objects that are in another memory location. C/C++ structs(classes) on the other hand represents a memory layout so an array of "objects" would be laid out as a single contigious block of memory. (and this works well for a GPU) There is/was a propsal for Java for value types but it hasn't yet been approved. https://openjdk.java.net/jeps/169 https://openjdk.java.net/jeps/169 (As a side note, C# classes are used as refernces like in Java but they also have "struct" in C# that behaves like C structs, and this is what the Unity boost compiler leverages and could be used for a GPU variant with less restrictions)
    - pjmlp 7y ago
      Not necessarily, although the language does not guarantee it, flattening to structs does happen in some JITs. Grated it isn't perfect, hence the value types proposal. Here is one of the CUDA commercial offerings for Java and .NET, https://developer.nvidia.com/altimesh-hybridizer https://developer.nvidia.com/altimesh-hybridizer
      - kllrnohj 7y ago
        > Not necessarily, although the language does not guarantee it, flattening to structs does happen in some JITs. Only if escape analysis says it didn't go anywhere. Which works for iterators but it's not going to work for input & output arrays to this system, especially since it's asynchronous meaning it did escape the stack frame.
imtringued 7y ago
CPUs are like taxis and GPUs are like buses. You have 64 people sitting in your bus all taking the same path. As soon as two people need to arrive at two different destinations (branching) this whole system breaks down. Eventually the bus must drive to all destinations. Then there is the time wasted stopping to load passengers. To efficiently utilize the bus you must load 64 people at the same. If you were to let people hail the bus like a taxi it would constantly have to stop to just load a single passenger while the other 63 people in the bus have to wait for that single person (random memory access). A pointer can point to object of any arbitrary sub type in any memory location. Suddenly you have both branching and random memory access. If your program is branching heavily then you may end up with 1 piece of data being computed by a core that can handle 64 pieces of data at once and random memory access kills you because GPUs don't have huge caches like CPUs. The way GPUs deal with high latency is that they simply batch a large amount of work and switch to a different "thread" during memory access. If all your threads are busy loading from memory then you won't see any speedups. Finally GPUs do not actually have 4096 cores. The RX Vega 64 only has 64 cores and those cores only run at around 1.3GHz. A Ryzen with 8 cores running at 4GHz with out of order execution can trivially outperform a GPU that is only using 1.6% (1/64) of its theoretical performance.
- ggm 7y ago
  Analogies break down or.. is this actually within the analogy? Some buses, they permit small variances from the set route to deal with specific issues which can be done en-route. The simplest is "drop me off anywhere on the route, but only pick up on demand at scheduled stops" The more complex is "wear one diversion every now and then" Some small buses actually self-optimize to a locally efficient route. Post-Buses in rural locations. It is surely possible GPU can segment, into small disjoint sets of "like" and so continue to offer parallelism, but at reduced intensity? Or, have point inefficiencies in the computation, toward the overall goal of "all alike"
  - dsun180 7y ago
    Yes. GPUs also do not have tires. Who cares?

AnimalMuppet 7y ago

Other way around - GPUs are not suitable for Java. Java is a dynamically-allocated, garbage-collected language. So far as I know, GPU code doesn't have the ability to dynamically allocate memory. (I mean, I guess you could, but you'd probably need one memory pool per parallel computational unit. Otherwise, you'd need a global lock on the global memory pool.) Java is a general programming language. GPUs are not general-purpose processors. That's off the top of my head. There may be other reasons as well.

pjmlp 7y ago
CUDA can dynamically allocate, more recente versions even support unified shared memory blocks with the host processor. It is also possible to do native memory allocation in Java. Imposing restrictions on a Java subset is hardly any different of imposing restrictions on C and C++, which apparently is ok to do.
- whizzter 7y ago
  The big difference that makes Java cumbersome is that you always have object references (ie pointers), thus an array of objects is really an array of pointers to objects that are in another memory location. C/C++ structs(classes) on the other hand represents a memory layout so an array of "objects" would be laid out as a single contigious block of memory. (and this works well for a GPU) There is/was a propsal for Java for value types but it hasn't yet been approved. https://openjdk.java.net/jeps/169 https://openjdk.java.net/jeps/169 (As a side note, C# classes are used as refernces like in Java but they also have "struct" in C# that behaves like C structs, and this is what the Unity boost compiler leverages and could be used for a GPU variant with less restrictions)
  - pjmlp 7y ago
    Not necessarily, although the language does not guarantee it, flattening to structs does happen in some JITs. Grated it isn't perfect, hence the value types proposal. Here is one of the CUDA commercial offerings for Java and .NET, https://developer.nvidia.com/altimesh-hybridizer https://developer.nvidia.com/altimesh-hybridizer
    - kllrnohj 7y ago
      > Not necessarily, although the language does not guarantee it, flattening to structs does happen in some JITs. Only if escape analysis says it didn't go anywhere. Which works for iterators but it's not going to work for input & output arrays to this system, especially since it's asynchronous meaning it did escape the stack frame.

imtringued 7y ago

CPUs are like taxis and GPUs are like buses. You have 64 people sitting in your bus all taking the same path. As soon as two people need to arrive at two different destinations (branching) this whole system breaks down. Eventually the bus must drive to all destinations. Then there is the time wasted stopping to load passengers. To efficiently utilize the bus you must load 64 people at the same. If you were to let people hail the bus like a taxi it would constantly have to stop to just load a single passenger while the other 63 people in the bus have to wait for that single person (random memory access). A pointer can point to object of any arbitrary sub type in any memory location. Suddenly you have both branching and random memory access. If your program is branching heavily then you may end up with 1 piece of data being computed by a core that can handle 64 pieces of data at once and random memory access kills you because GPUs don't have huge caches like CPUs. The way GPUs deal with high latency is that they simply batch a large amount of work and switch to a different "thread" during memory access. If all your threads are busy loading from memory then you won't see any speedups. Finally GPUs do not actually have 4096 cores. The RX Vega 64 only has 64 cores and those cores only run at around 1.3GHz. A Ryzen with 8 cores running at 4GHz with out of order execution can trivially outperform a GPU that is only using 1.6% (1/64) of its theoretical performance.

ggm 7y ago
Analogies break down or.. is this actually within the analogy? Some buses, they permit small variances from the set route to deal with specific issues which can be done en-route. The simplest is "drop me off anywhere on the route, but only pick up on demand at scheduled stops" The more complex is "wear one diversion every now and then" Some small buses actually self-optimize to a locally efficient route. Post-Buses in rural locations. It is surely possible GPU can segment, into small disjoint sets of "like" and so continue to offer parallelism, but at reduced intensity? Or, have point inefficiencies in the computation, toward the overall goal of "all alike"
- dsun180 7y ago
  Yes. GPUs also do not have tires. Who cares?