4 ms·

Things are changing extremely rapidly. Researchers (especially in parallel computing) are already figuring out how to port all sorts of algorithms over to the

by apu 17y ago

Things are changing extremely rapidly. Researchers (especially in parallel computing) are already figuring out how to port all sorts of algorithms over to the GPU -- and not just scientific or graphics code -- we're talking general purpose algorithms like sorting, ranking, cryptography, networking, database operations, etc: http://gpgpu.org/2009/04/13/efficient-acceleration-of-asymmetric-cryptography-on-graphics-hardware http://gpgpu.org/2009/04/13/efficient-acceleration-of-asymme... http://gpgpu.org/2009/04/28/fast-and-scalable-list-ranking-on-the-gpu http://gpgpu.org/2009/04/28/fast-and-scalable-list-ranking-o... http://gpgpu.org/2008/12/11/wait-free-programming-for-general-purpose-computations-on-graphics-processors http://gpgpu.org/2008/12/11/wait-free-programming-for-genera... http://gpgpu.org/2008/10/26/gnort-high-performance-network-intrusion-detection-using-graphics-processors http://gpgpu.org/2008/10/26/gnort-high-performance-network-i... http://gpgpu.org/2008/05/25/a-fast-similarity-join-algorithm-using-graphics-processing-units http://gpgpu.org/2008/05/25/a-fast-similarity-join-algorithm... The biggest obstacle to preventing widespread adoption of GPGPU techniques today is that things are still in so much flux that mature, stable APIs have not yet emerged. There is fierce competition between Intel (Larrabee), Apple/AMD (Grand Central/OpenCL), and NVIDIA (CUDA) to get their respective APIs to be the dominant ones, and no one's yet come up with a mature wrapper API that can target any of them. This is in large part because the hardware itself is still in flux, morphing away from the graphics-specific pipeline of 10 years ago and into a general purpose SIMD architecture. This has meant a change in the kinds of operations allowed on the GPU. For example, branches used to be a big no-no in graphics hardware: the performance hit caused by a branch was disastrous on a massively-multicore system. However, all hardware makers are now adding supporting for branches, because it's almost impossible to port a lot of CPU algorithms without them. In any case, I'm confident that within a year or two, things will have settled down somewhat, at which point there's going to be a mad dash by developers to start using GPUs (probably systems- and library-level developers rather than application-programmers).

jacquesm 17y ago
Well, count me in, as far as I'm concerned the mad dash has already begun. I've decided to splurge on the largest CUDA capable card that will fit this machine without blowing up the power supply. Stuff like this is still cutting edge but will be mainstream soon, the time to gain experience is today. Not to wait.
scott_s 17y ago
I am a researcher in parallel computing. All of these papers represent good work and are contributions to the field, but they still obey the limited model I was referring to: ship a large amount of data to the GPU, do data parallel computations on that data, and ship the results back. As I pointed out in a post below, this model does not work well when you only have small quantities of data at a time, yet there is parallelism to exploit.
- jacquesm 17y ago
  The basic metric for this kind of comparison is how long can you get your 'compute' node to work on a part of a problem without any new input data and without any intermediate results that need posting for other parts of the code to continue (rendez-vous I believe these are called). The longer that time the better suited the problem is for a massive parallel solution. If the time is low relative to the IO that needs to be done then you'll find very soon that the bus that carries data between the host CPU and the number cruncher is the bottle-neck.
  - skermes 17y ago
    The big problem is that you can work with some fairly large amounts of data and still have the bus be your bottleneck. I was doing some GPU work a few summers ago that focused on matrix multiplications. We were sending matrices with 8k numbers on a side to the GPU for multiplication and still ending up with the bus being the slowest part of the computation.
    - Retric 17y ago
      How much RAM was on the card? 64bit numbers * 8000^2 = 512 MB. Granted today you can have 4GB per card, but back then you where probably stuck with a fraction of that. Still, PCIe 2.0 x16 is limited to 8GByte/s so I guess the real question is how many matrixes where you multiplying?
- apu 17y ago
  Ah sorry, I didn't realize that =) I'll defer to your judgment about this stuff, then. (I'm a PhD student in computer vision, and we've recently been using a GPU SVM library that's been amazing for cutting down our processing times, so I guess I've been a little dazzled by this stuff.) Anyways, since you're in this field, what's your feeling about the future of parallel computing, with regards to the different vendors? Which of CUDA/OpenCL/Larrabee will win out? Or none of the above? When will APIs settle down?
codedivine 17y ago
GPUs are not a solution to every problem. There are several problems currently: a) Explicit data copy required to and from GPU makes GPGPU useless for many problems. b) GPUs are not as general purpose as CPUs. For example, no indirect jumps (goto address x) is allowed so no real subroutine calls. And no recursion. c) GPUs are not good at branching. The toll on branching can be heavy. d) Caches are fairly small. The on-chip shared memory, which is essentially a software controlled cache is a good idea in some cases but its still of a very small size. Well the thing is that the computational peak power of the GPUs is not free. It comes at the cost of reduced flexibility. Nothing is free in computer architecture.

by apu 17y ago

jacquesm 17y ago

Well, count me in, as far as I'm concerned the mad dash has already begun. I've decided to splurge on the largest CUDA capable card that will fit this machine without blowing up the power supply. Stuff like this is still cutting edge but will be mainstream soon, the time to gain experience is today. Not to wait.

scott_s 17y ago

I am a researcher in parallel computing. All of these papers represent good work and are contributions to the field, but they still obey the limited model I was referring to: ship a large amount of data to the GPU, do data parallel computations on that data, and ship the results back. As I pointed out in a post below, this model does not work well when you only have small quantities of data at a time, yet there is parallelism to exploit.

jacquesm 17y ago
The basic metric for this kind of comparison is how long can you get your 'compute' node to work on a part of a problem without any new input data and without any intermediate results that need posting for other parts of the code to continue (rendez-vous I believe these are called). The longer that time the better suited the problem is for a massive parallel solution. If the time is low relative to the IO that needs to be done then you'll find very soon that the bus that carries data between the host CPU and the number cruncher is the bottle-neck.
- skermes 17y ago
  The big problem is that you can work with some fairly large amounts of data and still have the bus be your bottleneck. I was doing some GPU work a few summers ago that focused on matrix multiplications. We were sending matrices with 8k numbers on a side to the GPU for multiplication and still ending up with the bus being the slowest part of the computation.
  - Retric 17y ago
    How much RAM was on the card? 64bit numbers * 8000^2 = 512 MB. Granted today you can have 4GB per card, but back then you where probably stuck with a fraction of that. Still, PCIe 2.0 x16 is limited to 8GByte/s so I guess the real question is how many matrixes where you multiplying?
apu 17y ago
Ah sorry, I didn't realize that =) I'll defer to your judgment about this stuff, then. (I'm a PhD student in computer vision, and we've recently been using a GPU SVM library that's been amazing for cutting down our processing times, so I guess I've been a little dazzled by this stuff.) Anyways, since you're in this field, what's your feeling about the future of parallel computing, with regards to the different vendors? Which of CUDA/OpenCL/Larrabee will win out? Or none of the above? When will APIs settle down?

codedivine 17y ago

GPUs are not a solution to every problem. There are several problems currently: a) Explicit data copy required to and from GPU makes GPGPU useless for many problems. b) GPUs are not as general purpose as CPUs. For example, no indirect jumps (goto address x) is allowed so no real subroutine calls. And no recursion. c) GPUs are not good at branching. The toll on branching can be heavy. d) Caches are fairly small. The on-chip shared memory, which is essentially a software controlled cache is a good idea in some cases but its still of a very small size. Well the thing is that the computational peak power of the GPUs is not free. It comes at the cost of reduced flexibility. Nothing is free in computer architecture.