5 ms·

I find the article a bit overoptimistic about the future and applicability of heterogenous computing. The problem with delegating tasks to GPGPU's is, that GPG

I find the article a bit overoptimistic about the future and applicability of heterogenous computing. The problem with delegating tasks to GPGPU's is, that GPGPU cores can do only a very limited subset of what a CPU can do. First of all, each GPGPU core has only a small amount of fast local memory available. In most applications, tasks need to be able to access databases, large in memory datastructures and what not. The GPGPU architecture is not designed for that. When data outside the core's local memory is needed, data transfer to and from individual cores is costly and when too much of that has to happen, the benefit of the many GPGPU cores rapidly disappears. Second, a GPGPU core does not have access to other components, e.g. IO controllers. So it can't directly read from disk or write to the network. This again limits the types of work that a GPGPU core can do by a lot. For all network and disk access it needs to communicate with the CPU, and again, this communication is slow. And finally there's a difficulty of another nature. OpenCL or CUDA does not integrate nicely with higher level languages such as Java. The article mentions Aparapi, which is one solution that tries to deal with this by compiling Java code to OpenCL. The trouble with this is that Java code is living in another world as OpenCL; code that works well for Java might perform horribly on the GPGPU, because the GPGPU's architecture with its memory locality is so different. So to write Java code that will be compiled to well performing OpenCL, you have to code it with OpenCL in mind, and you have to be aware of the manner in which Aparapi will convert your code to OpenCL. This defeats the purpose of Aparapi; it might very well be easier to just write your code in OpenCL so at least you have a clear view what is going on.

venomsnake 11y ago
So GPGPU is useful for computation intensive/ no branching map reduce of vectors that can be fit into GPU memory. Can we have a smart VM that profiles the code and sees that kind of code and put it on GPU?
- exDM69 11y ago
  > So GPGPU is useful for computation intensive/ no branching map reduce of vectors that can be fit into GPU memory. Modern GPUs can do full dynamic branching and process data structures that aren't flat (ie. vectors/matrices). It's not restricted to such simple tasks any more. I don't think we'll ever have VMs that are smart enough to do decisions like this better than an engineer can. Or even close. What we really need is smart engineers who better understand what the technology is suitable for and can make good decisions on when to employ it.
  - michaelt 11y ago
    That's interesting to hear! I took the udacity CUDA course a few years ago, and at that time they were very keen on memory access coalescing, and if a warp had two control paths they were executed sequentially. As such I didn't think I could figure out how to make use of it for my applications (merging sorted lists of integers, dijkstra's algorithm etc) as typically you're branching and jumping around memory all over the place. Can you suggest anywhere I can learn about these advances that mean the same rules don't apply?
- sklogic 11y ago
  Some forms of vectorisation compiler passes (e.g., a polyhedral analysis) can detect such patterns statically. But, so far, they only work reliably in cases where it is equally easy to offload the processing to a GPU manually.
kctess5 11y ago
I think there may be solutions to some of these things. Firstly, it sounds they are envisioning pipelined algorithms where stages are run on the best compute device. The CPU pipeline would presumably handle all asynchronous Apis and only call the pipelined algorithms once the data was available. It would be much easier to parallelize the code that can be easily be parallelized if the gpu context is the same as the cpu context. The second part I think is up to compilers. If we can make better compilers for gpu code which integrate more nicely with cpu code then the super low level programming won't be as necessary. GPUs haven't been in the limelight nearly as long as CPUs, so it's understandable that most modern languages are made for CPU code. I'd personally like to see LLVM and PTX playing together nicely in Rust. Humans will still have to consider the differences in optimal memory access / branching logic patterns, but that's understandable. We already do that - with the CPU in mind - so it will just take a bit of extra learning to consider the GPU.
creshal 11y ago
> First of all, each GPGPU core has only a small amount of fast local memory available. > Second, a GPGPU core does not have access to other components, e.g. IO controllers. Didn't AMD promise to solve both with HSA, which should allow (integrated) GPGPU cores to share all system resources with the rest of the CPU?
- misja111 11y ago
  Yes, HSA was designed as a solution for these problems. It looks like a good solution, however as far as I know there is not any hardware platform implementing the HSA standards yet. Also Intel is unfortunately not part of the HSA Alliance.
  - markus2012 11y ago
    It seems AMD's Kaveri CPU implements HSA. Some HSA benchmarks from AnandTech: http://www.tomshardware.com/reviews/amd-a10-7800-kaveri-apu-efficiency,3899-5.html http://www.tomshardware.com/reviews/amd-a10-7800-kaveri-apu-... Note: the tests seem to use OpenCL and while they mention, "Looking at the HSA-based results..." I find it strange they are comparing the numbers against an Intel Core i3. It might be more interesting to compare OpenCL HSA vs OpenCL non-HSA.
  - mtgx 11y ago
    I think this article just said they now are.
exDM69 11y ago
I definitely agree that GPGPU has limitations in what it can do, but there's a few details that are a bit skewed in your analysis. Perhaps your facts aren't quite up to date (by, say, 5 years or so) First: memory. A consumer GPU these days has 3 GiB of memory and a compute/viz GPU has up to 12 GiB. The memory bandwidth is about 300 GiB/s compared to 50 GiB per second for the CPU-DRAM. In other words: in memory intensive applications, the GPU wins. There's no "local memory" in modern GPU architectures (unlike 5 or so years ago), they have L1/L2 caches not unlike the CPU. > For all network and disk access it needs to communicate with the CPU, and again, this communication is slow. This is certainly true that the GPU can't do i/o, but "slow" is an ambiguous term. Yes, it is high latency and difficult to orchestrate effectively but the memory bandwidth of a PCI-e bus is 16 GiB/s (v3.x) to 32 GiB/s (v4.x), that's 30-60% of the bandwidth between the CPU and DRAM. GPUs can handle complex in-memory data structures, and are very effective in processing e.g. tree structures. There's even been applications of GPUs to databases (the PG-storm backend for postgres, for example). Modern GPUs are entirely different beasts to what the matrix and vector batch processing chips were a few years ago were. Yes, it's a big shift in paradigms and requires some hard software engineering but the benefits are there. Case in point: the "deep learning revolution", in particular training the networks, would not have happened were it not for GPU tech.

venomsnake 11y ago

So GPGPU is useful for computation intensive/ no branching map reduce of vectors that can be fit into GPU memory. Can we have a smart VM that profiles the code and sees that kind of code and put it on GPU?

exDM69 11y ago
> So GPGPU is useful for computation intensive/ no branching map reduce of vectors that can be fit into GPU memory. Modern GPUs can do full dynamic branching and process data structures that aren't flat (ie. vectors/matrices). It's not restricted to such simple tasks any more. I don't think we'll ever have VMs that are smart enough to do decisions like this better than an engineer can. Or even close. What we really need is smart engineers who better understand what the technology is suitable for and can make good decisions on when to employ it.
- michaelt 11y ago
  That's interesting to hear! I took the udacity CUDA course a few years ago, and at that time they were very keen on memory access coalescing, and if a warp had two control paths they were executed sequentially. As such I didn't think I could figure out how to make use of it for my applications (merging sorted lists of integers, dijkstra's algorithm etc) as typically you're branching and jumping around memory all over the place. Can you suggest anywhere I can learn about these advances that mean the same rules don't apply?
sklogic 11y ago
Some forms of vectorisation compiler passes (e.g., a polyhedral analysis) can detect such patterns statically. But, so far, they only work reliably in cases where it is equally easy to offload the processing to a GPU manually.

kctess5 11y ago

I think there may be solutions to some of these things. Firstly, it sounds they are envisioning pipelined algorithms where stages are run on the best compute device. The CPU pipeline would presumably handle all asynchronous Apis and only call the pipelined algorithms once the data was available. It would be much easier to parallelize the code that can be easily be parallelized if the gpu context is the same as the cpu context. The second part I think is up to compilers. If we can make better compilers for gpu code which integrate more nicely with cpu code then the super low level programming won't be as necessary. GPUs haven't been in the limelight nearly as long as CPUs, so it's understandable that most modern languages are made for CPU code. I'd personally like to see LLVM and PTX playing together nicely in Rust. Humans will still have to consider the differences in optimal memory access / branching logic patterns, but that's understandable. We already do that - with the CPU in mind - so it will just take a bit of extra learning to consider the GPU.

creshal 11y ago

> First of all, each GPGPU core has only a small amount of fast local memory available. > Second, a GPGPU core does not have access to other components, e.g. IO controllers. Didn't AMD promise to solve both with HSA, which should allow (integrated) GPGPU cores to share all system resources with the rest of the CPU?

misja111 11y ago
Yes, HSA was designed as a solution for these problems. It looks like a good solution, however as far as I know there is not any hardware platform implementing the HSA standards yet. Also Intel is unfortunately not part of the HSA Alliance.
- markus2012 11y ago
  It seems AMD's Kaveri CPU implements HSA. Some HSA benchmarks from AnandTech: http://www.tomshardware.com/reviews/amd-a10-7800-kaveri-apu-efficiency,3899-5.html http://www.tomshardware.com/reviews/amd-a10-7800-kaveri-apu-... Note: the tests seem to use OpenCL and while they mention, "Looking at the HSA-based results..." I find it strange they are comparing the numbers against an Intel Core i3. It might be more interesting to compare OpenCL HSA vs OpenCL non-HSA.
- mtgx 11y ago
  I think this article just said they now are.

exDM69 11y ago

I definitely agree that GPGPU has limitations in what it can do, but there's a few details that are a bit skewed in your analysis. Perhaps your facts aren't quite up to date (by, say, 5 years or so) First: memory. A consumer GPU these days has 3 GiB of memory and a compute/viz GPU has up to 12 GiB. The memory bandwidth is about 300 GiB/s compared to 50 GiB per second for the CPU-DRAM. In other words: in memory intensive applications, the GPU wins. There's no "local memory" in modern GPU architectures (unlike 5 or so years ago), they have L1/L2 caches not unlike the CPU. > For all network and disk access it needs to communicate with the CPU, and again, this communication is slow. This is certainly true that the GPU can't do i/o, but "slow" is an ambiguous term. Yes, it is high latency and difficult to orchestrate effectively but the memory bandwidth of a PCI-e bus is 16 GiB/s (v3.x) to 32 GiB/s (v4.x), that's 30-60% of the bandwidth between the CPU and DRAM. GPUs can handle complex in-memory data structures, and are very effective in processing e.g. tree structures. There's even been applications of GPUs to databases (the PG-storm backend for postgres, for example). Modern GPUs are entirely different beasts to what the matrix and vector batch processing chips were a few years ago were. Yes, it's a big shift in paradigms and requires some hard software engineering but the benefits are there. Case in point: the "deep learning revolution", in particular training the networks, would not have happened were it not for GPU tech.