3 ms·

Those extra cores are designed to be good for streaming parallel data computations, not "generic code" - you can't even run a hash table performantly on a GPU d

Those extra cores are designed to be good for streaming parallel data computations, not "generic code" - you can't even run a hash table performantly on a GPU due to the random pointer accesses.

pandaman 13y ago
Random pointer access is slow everywhere. However, a massively parallel GPU code will likely beat multi-thread CPU code there. A GPU can switch threads on a cache miss, something few CPUs can do. Even the CPUs that do this switch between 2-8 threads (it's called SMT architecture, for example, popular Intel Pentium/Core CPUs have at most 2 hardware threads per core, Xenon Phi has 4). GPUs have dozens of threads per core and, often, more cores than CPUs. Combined with the fact that a GPU thread is much wider than a CPU this allows a GPU to saturate memory bandwidth even on a pointer chasing code. The same code causes the CPU to stall and underutilize its memory bus (as soon as the few threads hit cache miss there is nothing to do, software multithreading cannot switch between threads that are waiting for data). Of course, as I said, it's still not efficient even on GPU (most of the bandwidth is used to move unneeded data as you only need one word from the whole cache line) and you still need to write massively parallel code.
- weland 13y ago
  > However, a massively parallel GPU code will likely beat multi-thread CPU code there. Yes, but if you actually try to do it, you'll find that "massively parallel" code can be written only for certain problems that lend themselves to being, well, "massively parallel". I've seen people struggling with GPGPUs ever since they appeared (at a certain point in my career I was involved in heavy number-crunching research). Six years later, there are still a lot of real-life problems that can't be put in a GPGPU-useful framework; in fact, coming up with good models so that you can reason if a given problem would ever lend itself to being solve like that is one of the more important progresses of the scientific community. Look, GPGPUs are cool and all, and useful for certain classes of problems, but let's not do the whole Itanium thing all over again, please.
  - pandaman 13y ago
    Indeed, there are problems that cannot be solved in a parallel fashion. If you are stuck with such a problem you are screwed no matter what is your architecture. A whole lot of real-life stuff is either parallel or does not need heavy computing. This is why people started building massively parallel computers well before the last 6 years[0]. [0] http://en.wikipedia.org/wiki/History_of_supercomputing http://en.wikipedia.org/wiki/History_of_supercomputing
    - weland 13y ago
      > A whole lot of real-life stuff is either parallel or does not need heavy computing. This is why people started building massively parallel computers well before the last 6 years[0]. Exactly. A lot of the GPGPU pitch is easily refuted with some hindsight.
  - maaku 13y ago
    A lot of that struggling could have been avoided if the proper tools were available. For example, the compiler could automatically include two code paths - one AMD64, one OpenCL - and use the best one or switch between the two based on performance or runtime considerations. Language-level improvements (like what Erlang and Haskell have) could make writing parallel code much easier. I said that APUs are the future of computing not because I'm favoring parallel applications, but because boring old manufacturing constraints are making greater single-threaded performance more difficult to achieve. What we have now is about as good as it's going to get, at least until we see 3D chips or molecular nanotechnology.

Those extra cores are designed to be good for streaming parallel data computations, not "generic code" - you can't even run a hash table performantly on a GPU due to the random pointer accesses.

pandaman 13y ago

Random pointer access is slow everywhere. However, a massively parallel GPU code will likely beat multi-thread CPU code there. A GPU can switch threads on a cache miss, something few CPUs can do. Even the CPUs that do this switch between 2-8 threads (it's called SMT architecture, for example, popular Intel Pentium/Core CPUs have at most 2 hardware threads per core, Xenon Phi has 4). GPUs have dozens of threads per core and, often, more cores than CPUs. Combined with the fact that a GPU thread is much wider than a CPU this allows a GPU to saturate memory bandwidth even on a pointer chasing code. The same code causes the CPU to stall and underutilize its memory bus (as soon as the few threads hit cache miss there is nothing to do, software multithreading cannot switch between threads that are waiting for data). Of course, as I said, it's still not efficient even on GPU (most of the bandwidth is used to move unneeded data as you only need one word from the whole cache line) and you still need to write massively parallel code.

weland 13y ago
> However, a massively parallel GPU code will likely beat multi-thread CPU code there. Yes, but if you actually try to do it, you'll find that "massively parallel" code can be written only for certain problems that lend themselves to being, well, "massively parallel". I've seen people struggling with GPGPUs ever since they appeared (at a certain point in my career I was involved in heavy number-crunching research). Six years later, there are still a lot of real-life problems that can't be put in a GPGPU-useful framework; in fact, coming up with good models so that you can reason if a given problem would ever lend itself to being solve like that is one of the more important progresses of the scientific community. Look, GPGPUs are cool and all, and useful for certain classes of problems, but let's not do the whole Itanium thing all over again, please.
- pandaman 13y ago
  Indeed, there are problems that cannot be solved in a parallel fashion. If you are stuck with such a problem you are screwed no matter what is your architecture. A whole lot of real-life stuff is either parallel or does not need heavy computing. This is why people started building massively parallel computers well before the last 6 years[0]. [0] http://en.wikipedia.org/wiki/History_of_supercomputing http://en.wikipedia.org/wiki/History_of_supercomputing
  - weland 13y ago
    > A whole lot of real-life stuff is either parallel or does not need heavy computing. This is why people started building massively parallel computers well before the last 6 years[0]. Exactly. A lot of the GPGPU pitch is easily refuted with some hindsight.
- maaku 13y ago
  A lot of that struggling could have been avoided if the proper tools were available. For example, the compiler could automatically include two code paths - one AMD64, one OpenCL - and use the best one or switch between the two based on performance or runtime considerations. Language-level improvements (like what Erlang and Haskell have) could make writing parallel code much easier. I said that APUs are the future of computing not because I'm favoring parallel applications, but because boring old manufacturing constraints are making greater single-threaded performance more difficult to achieve. What we have now is about as good as it's going to get, at least until we see 3D chips or molecular nanotechnology.