5 ms·

> You see, the (not so) dirty secret of GCN is that from a very real perspective, the best way to think about it is that it's a bunch of CPU cores with extremel

> You see, the (not so) dirty secret of GCN is that from a very real perspective, the best way to think about it is that it's a bunch of CPU cores with extremely wide masked SIMD units. This is not quite doing it justice. GPU threads allow for branching (which thus includes things like early return). They do loose performance because inactive branches are sleeping (although I didn't test this with Volta and onward, which is supposedly brings some improvements), but that's way better than just breaking vectorization outright. This makes it much easier to port some naive (or just cache-optimized) piece of numerics code to GPU rather than trying to vectorize it on CPU.

gpderetta 8y ago
How's that different from masking (i.e per-line predication)?
- m_mueller 8y ago
  the difference is that for masking you have to rely on the compiler to do-the-right-thing[TM] OR you have to program it yourself manually (which is quite annoying). With GPU threads the scheduling of branches is hardware supported, so it's much easier to map your scalar code to it as long as it's sufficiently data parallel.
  - atq2119 8y ago
    That's the thing though: for AMD GCN, the compilers do have to do the right thing. You can look at the actual hardware ISA, it's all documented e.g. here: https://rocm-documentation.readthedocs.io/en/latest/GCN_ISA_Manuals/GCN-ISA-Manuals.html https://rocm-documentation.readthedocs.io/en/latest/GCN_ISA_..., and both the HCC compiler and Mesa's OpenGL and Vulkan implementations are open-source and based on LLVM, where the backend is even upstream, so you can take a look for yourself. Essentially everything those compilers do would be possible on a CPU with masked SIMD instructions (e.g. AVX512). In fact, that's basically how ISPC works. The one exception is that GCN has much more powerful scatter/gather support than CPUs do (and graphics-focused stuff like texture sampling hardware). But as far as the execution model is concerned, it's really the same.
    - m_mueller 8y ago
      I see, that's interesting. So essentially, AMD's hardware schedulers are not concerned with branching the same way Nvidia seems to be doing it?
      - dragontamer 8y ago
        Pascal does branching in a similar manner. Volta / Turing does branching per-PC now, but it is opt-in optional. In any case, I expect Volta / Turing to prefer the old SIMD model of compute. The only reason Volta / Turing supports per-PC branching is to handle some multi-thread communication issues. The execution units are still SIMD, so you take a heavy penalty going to the per-PC model. PS: I haven't actually programmed Volta/Turing, its just my understanding from the white paper.
- dragontamer 8y ago
  > How's that different from masking (i.e per-line predication)? AMD GCN has hardware accelerated masking + stacks for this purpose. AMD's GPU is basically a combination of masking + fork/join commands, to split up wavefronts. But its really just hardware accelerated masks. You sometimes need to store the masks + program counters in a stack to allow for divergent threads, and then pop the masks off and execute older commands... (Complex if / then / else branches where every thread takes a different set of branches).

gpderetta 8y ago

How's that different from masking (i.e per-line predication)?

m_mueller 8y ago
the difference is that for masking you have to rely on the compiler to do-the-right-thing[TM] OR you have to program it yourself manually (which is quite annoying). With GPU threads the scheduling of branches is hardware supported, so it's much easier to map your scalar code to it as long as it's sufficiently data parallel.
- atq2119 8y ago
  That's the thing though: for AMD GCN, the compilers do have to do the right thing. You can look at the actual hardware ISA, it's all documented e.g. here: https://rocm-documentation.readthedocs.io/en/latest/GCN_ISA_Manuals/GCN-ISA-Manuals.html https://rocm-documentation.readthedocs.io/en/latest/GCN_ISA_..., and both the HCC compiler and Mesa's OpenGL and Vulkan implementations are open-source and based on LLVM, where the backend is even upstream, so you can take a look for yourself. Essentially everything those compilers do would be possible on a CPU with masked SIMD instructions (e.g. AVX512). In fact, that's basically how ISPC works. The one exception is that GCN has much more powerful scatter/gather support than CPUs do (and graphics-focused stuff like texture sampling hardware). But as far as the execution model is concerned, it's really the same.
  - m_mueller 8y ago
    I see, that's interesting. So essentially, AMD's hardware schedulers are not concerned with branching the same way Nvidia seems to be doing it?
    - dragontamer 8y ago
      Pascal does branching in a similar manner. Volta / Turing does branching per-PC now, but it is opt-in optional. In any case, I expect Volta / Turing to prefer the old SIMD model of compute. The only reason Volta / Turing supports per-PC branching is to handle some multi-thread communication issues. The execution units are still SIMD, so you take a heavy penalty going to the per-PC model. PS: I haven't actually programmed Volta/Turing, its just my understanding from the white paper.
dragontamer 8y ago
> How's that different from masking (i.e per-line predication)? AMD GCN has hardware accelerated masking + stacks for this purpose. AMD's GPU is basically a combination of masking + fork/join commands, to split up wavefronts. But its really just hardware accelerated masks. You sometimes need to store the masks + program counters in a stack to allow for divergent threads, and then pop the masks off and execute older commands... (Complex if / then / else branches where every thread takes a different set of branches).