4 ms·

So the funny thing is that gcc's backend for GCN is actually much closer to writing for standard CPUs than any other way of programming GPUs, because you direct

So the funny thing is that gcc's backend for GCN is actually much closer to writing for standard CPUs than any other way of programming GPUs, because you directly program it at the wave-level. You see, the (not so) dirty secret of GCN is that from a very real perspective, the best way to think about it is that it's a bunch of CPU cores with extremely wide masked SIMD units. Almost all GPU programming languages obscure that fact, and their compilers do magic behind the scenes to make it appear as if you were programming individual threads without SIMD. GCC is the odd one out, which is really quite fascinating.

mappu 8y ago
Further reading: https://gcc.gnu.org/wiki/cauldron2018?action=AttachFile&do=get&target=amd-gcn-cauldron-presentation-2018.pdf https://gcc.gnu.org/wiki/cauldron2018?action=AttachFile&do=g... (e.g. slide 22)
elcritch 8y ago
Do the SIMD units have different memory blocks that while being controlled by one cpu core would make the super wide SIMD behave more like multiple threads with different memory units? Or at least act like it from a latency perspective?
- atq2119 8y ago
  I guess you're talking about scatter / gather? All vector memory instructions in the GCN ISA are scatter / gather-type instructions, i.e. you provide one pointer (or buffer index, or texture coordinates) per SIMD lane. Docs say that the performance depends on the actual distribution of pointers, i.e. for best performance you should ensure that the pointers are consecutive (use SoA instead of AoS layouts, etc.), but I imagine that the drop in performance is somewhat gradual and graceful as your pointers become more scattered. I don't think the details are really documented though, and I wouldn't be surprised if those details changed between GCN generations.
  - justinclift 8y ago
    Sounds like it'd be a useful benchmark to do + publish for people's reference. May also lead to some tuning choices upstream too, depending on the results. :)
- wongarsu 8y ago
  From a memory perspective you can view it as multiple threads. Caches are a bit more complicated (each "cpu core"/streaming processor has it's own L1 cache, and cache coherency between L1 caches isn't nessesarily guaranteed). Memory bandwidth and latency are still bottlenecks. The bigger implication of the SIMD design is that for any conditional all your "threads" should take the same route, otherwise you may end up computing both paths for every thread.
m_mueller 8y ago
> You see, the (not so) dirty secret of GCN is that from a very real perspective, the best way to think about it is that it's a bunch of CPU cores with extremely wide masked SIMD units. This is not quite doing it justice. GPU threads allow for branching (which thus includes things like early return). They do loose performance because inactive branches are sleeping (although I didn't test this with Volta and onward, which is supposedly brings some improvements), but that's way better than just breaking vectorization outright. This makes it much easier to port some naive (or just cache-optimized) piece of numerics code to GPU rather than trying to vectorize it on CPU.
- gpderetta 8y ago
  How's that different from masking (i.e per-line predication)?
  - m_mueller 8y ago
    the difference is that for masking you have to rely on the compiler to do-the-right-thing[TM] OR you have to program it yourself manually (which is quite annoying). With GPU threads the scheduling of branches is hardware supported, so it's much easier to map your scalar code to it as long as it's sufficiently data parallel.
    - atq2119 8y ago
      That's the thing though: for AMD GCN, the compilers do have to do the right thing. You can look at the actual hardware ISA, it's all documented e.g. here: https://rocm-documentation.readthedocs.io/en/latest/GCN_ISA_Manuals/GCN-ISA-Manuals.html https://rocm-documentation.readthedocs.io/en/latest/GCN_ISA_..., and both the HCC compiler and Mesa's OpenGL and Vulkan implementations are open-source and based on LLVM, where the backend is even upstream, so you can take a look for yourself. Essentially everything those compilers do would be possible on a CPU with masked SIMD instructions (e.g. AVX512). In fact, that's basically how ISPC works. The one exception is that GCN has much more powerful scatter/gather support than CPUs do (and graphics-focused stuff like texture sampling hardware). But as far as the execution model is concerned, it's really the same.

mappu 8y ago

Further reading: https://gcc.gnu.org/wiki/cauldron2018?action=AttachFile&do=get&target=amd-gcn-cauldron-presentation-2018.pdf https://gcc.gnu.org/wiki/cauldron2018?action=AttachFile&do=g... (e.g. slide 22)

elcritch 8y ago

Do the SIMD units have different memory blocks that while being controlled by one cpu core would make the super wide SIMD behave more like multiple threads with different memory units? Or at least act like it from a latency perspective?

atq2119 8y ago
I guess you're talking about scatter / gather? All vector memory instructions in the GCN ISA are scatter / gather-type instructions, i.e. you provide one pointer (or buffer index, or texture coordinates) per SIMD lane. Docs say that the performance depends on the actual distribution of pointers, i.e. for best performance you should ensure that the pointers are consecutive (use SoA instead of AoS layouts, etc.), but I imagine that the drop in performance is somewhat gradual and graceful as your pointers become more scattered. I don't think the details are really documented though, and I wouldn't be surprised if those details changed between GCN generations.
- justinclift 8y ago
  Sounds like it'd be a useful benchmark to do + publish for people's reference. May also lead to some tuning choices upstream too, depending on the results. :)
wongarsu 8y ago
From a memory perspective you can view it as multiple threads. Caches are a bit more complicated (each "cpu core"/streaming processor has it's own L1 cache, and cache coherency between L1 caches isn't nessesarily guaranteed). Memory bandwidth and latency are still bottlenecks. The bigger implication of the SIMD design is that for any conditional all your "threads" should take the same route, otherwise you may end up computing both paths for every thread.

m_mueller 8y ago

> You see, the (not so) dirty secret of GCN is that from a very real perspective, the best way to think about it is that it's a bunch of CPU cores with extremely wide masked SIMD units. This is not quite doing it justice. GPU threads allow for branching (which thus includes things like early return). They do loose performance because inactive branches are sleeping (although I didn't test this with Volta and onward, which is supposedly brings some improvements), but that's way better than just breaking vectorization outright. This makes it much easier to port some naive (or just cache-optimized) piece of numerics code to GPU rather than trying to vectorize it on CPU.

gpderetta 8y ago
How's that different from masking (i.e per-line predication)?
- m_mueller 8y ago
  the difference is that for masking you have to rely on the compiler to do-the-right-thing[TM] OR you have to program it yourself manually (which is quite annoying). With GPU threads the scheduling of branches is hardware supported, so it's much easier to map your scalar code to it as long as it's sufficiently data parallel.
  - atq2119 8y ago
    That's the thing though: for AMD GCN, the compilers do have to do the right thing. You can look at the actual hardware ISA, it's all documented e.g. here: https://rocm-documentation.readthedocs.io/en/latest/GCN_ISA_Manuals/GCN-ISA-Manuals.html https://rocm-documentation.readthedocs.io/en/latest/GCN_ISA_..., and both the HCC compiler and Mesa's OpenGL and Vulkan implementations are open-source and based on LLVM, where the backend is even upstream, so you can take a look for yourself. Essentially everything those compilers do would be possible on a CPU with masked SIMD instructions (e.g. AVX512). In fact, that's basically how ISPC works. The one exception is that GCN has much more powerful scatter/gather support than CPUs do (and graphics-focused stuff like texture sampling hardware). But as far as the execution model is concerned, it's really the same.