9 ms·

Why are Intel GPUs designed in such a way that typical GPU languages don’t fully exploit it? Is the new Xe architecture still SIMD?

by skavi 6y ago

pauljurczak 6y ago
The Intel Gen GPU architecture, which includes the newest incarnation Gen12 aka Xe, is a SIMD architecture as opposed to Nvidia and AMD SIMT architecture. The reasons are historical, i.e. CPU centric design: x86 CPU, Larrabee, Xeon-Phi, etc.
dragontamer 6y ago
OpenCL works on Intel GPUs, while CUDA doesn't because CUDA is an NVidia technology. > Is the new Xe architecture still SIMD? SIMD is... pretty much all GPUs do. There's a few scalar bits here and there to speed up if-statements and the like, but the entire point of a GPU is to build a machine for SIMD.
- oivey 6y ago
  OpenCL is basically dead at this point, too. The de facto standard is CUDA and there aren’t currently any real challengers. Maybe eventually AMD’s ROCm or Intel’s oneAPI will get traction.
  - pjmlp 6y ago
    For them to get traction, they need to invest in debugger tooling that allows the productivity as on CPUs, and to help language communities other than C and C++ to target GPGPUs. NVidia started doing both around CUDA 3.0, whereas Khronos, AMD and Intel only started paying attention that not everyone wanted to do printf() style debugging with a C dialect until it was too late to get people's attention back.
    - MaxBarraclough 6y ago
      AMD had a good Visual Studio plugin for OpenCL, complete with debugging support, although I believe it's since been discontinued.
  - profquail 6y ago
    oneAPI uses DPC++ (Data-Parallel C++), which is pretty much just SYCL, which itself is a C++ library on top of OpenCL. From my understanding, the Khronos group realized OpenCL 2.x was much too complicated so vendors just weren’t implementing it, or only implementing parts of it, so they came up with OpenCL 3.0 which is slimmed-down and much more modular. It’s hard to say how much adoption it’ll get, but with Intel focused on DPC++ and oneAPI now, there will definitely be more numerical software coming out in the next few years that compiles down to and runs on OpenCL. For example, Intel engineers are building a numpy clone on top of DPC++, so unlike regular numpy it’ll take advantage of multiple CPU cores: https://github.com/IntelPython/dpnp https://github.com/IntelPython/dpnp
    - astrange 6y ago
      > From my understanding, the Khronos group realized OpenCL 2.x was much too complicated so vendors just weren’t implementing it, or only implementing parts of it, so they came up with OpenCL 3.0 which is slimmed-down and much more modular. Something like this also happened to OpenGL 4.3. It added a compute shader extension which was essentially all of OpenCL again, except different, so you had 2x the implementation work. This is about when some people stopped implementing OpenGL.
      - TazeTSchnitzel 6y ago
        OpenGL compute shaders are a natural step if you have unified programmable shader cores, and less complicated than adding new pipeline stages for everything (tessellation shaders, geometry shaders, …). Khronos could have chosen only to add OpenCL integration, but OpenCL C is a very different language to GLSL, the memory model (among other things) is different, and so on. I don't see why video game developers should be forced to use OpenCL when they want to work with the outputs of OpenGL rendering passes, to produce inputs to OpenGL rendering passes, scheduled in OpenGL, to do things that don't fit neatly into vertex or fragment shaders?
      - pjmlp 6y ago
        On the contrary, for example on Android OpenGL ES you get compute shaders, but no OpenCL. By the way, the latest version is OpenGL 4.6, it is also available on the Switch.
  - gspr 6y ago
    I realize the size of the ecosystem is important, but if you're writing all the GPU code yourself, I wouldn't say OpenCL is dead. I recently had a chance to learn the basics for a work project, never having touched the field before. I picked OpenCL, because I knew I was writing all my non-BLAS code myself, and there's no way in hell I'll voluntarily lock myself into a closed ecosystem. (PS: CLBlast, which is different from CLblas, is a joy!) I was pleasantly surprised. I found OpenCL very nice to work with indeed! And my code runs on any modern GPU out there. I've tested it on Intel integrated GPUs, AMD GPUs, and Nvidia's fancy datacenter devices. And even CPUs. Seamlessly, through a runtime switch fully controlled by the application itself! Now, could I have gotten more performance out of CUDA? Yeah, I estimate about a factor 2. For the cost of tying myself to a proprietary, locked in technology from a hostile vendor, throwing out two major classes of devices, and losing the ability to test out code anywhere. Not worth it. I hope OpenCL has life in it still. The stuff I keep reading that CUDA is far easier to approach definitely did not ring true to this beginner.
- astrange 6y ago
  GPUs don't need to have SIMD instructions; if you give one a fully scalar program it just needs to run a lot of copies of it at once. Every architecture is different here, including within the same vendor.
  - dragontamer 6y ago
    > GPUs don't need to have SIMD instructions; Except NVidia Ampere (RTX 3xxx series) and AMD RDNA2 (Navi / 6xxx series) are both SIMD architectures with SIMD-instructions. And the #3 company: Intel, also has SIMD instructions. I know that some GPUs out there are VLIW or other weird architectures, but... the "big 3" are SIMD-based. > if you give one a fully scalar program it just needs to run a lot of copies of it at once. Its emulated on a SIMD processor. That SIMD processor will suffer branch-divergence as you traverse through if-statements and while-statements, because its physically SIMD. The compiler / programming model is scalar. But the assembly instructions are themselves vector. Yeah, NVidia now has per-SIMD core instruction pointers. But that doesn't mean that the hardware can physically execute different instructions: they're still all locked together with SIMD-style at the physical level.
    - my123 6y ago
      > with SIMD-instructions You assume so, for Volta onwards that's not true. Each SIMT thread on NVIDIA GPUs for Volta onwards has its own instruction pointer. And yes, it's scalar instructions at the ISA level.
      - dragontamer 6y ago
        > You assume so, for Volta onwards that's not true. https://arxiv.org/pdf/1804.06826.pdf https://arxiv.org/pdf/1804.06826.pdf NVidia's SASS on Volta is pretty clearly SIMD-instructions. The PTX "virtual machine assembly" is fully documented. SASS is not documented, but it ain't a secret either. SASS follows closely with PTX, with exception of some "barrier bits" that seem to track dependencies (probably compiler-managed read/write dependency chains). > Each SIMT thread on NVIDIA GPUs for Volta onwards has its own instruction pointer. And yes, it's scalar instructions at the ISA level. When your "scalar" instruction executes 32-threads in parallel (subject to an execution mask), that's called SIMD. The instruction pointer is to resolve deadlock conditions with locks, and the 32-wide SIMD cores will execute one-at-a-time to prevent deadlocks. But your goal as a GPU programmer is to get as much 32-wide execution going on as possible. 1-at-a-time serialization is very, very bad for performance. Its POSSIBLE to do on Volta, but highly recommended you stay away from that cornercase (you lose over 95% of your performance).
TinkersW 6y ago
The paper covers the reasons, and they aren't new or specific to Intel. There was a talk by Andrew Lauritzen basically saying the same thing a few years back. https://www.ea.com/seed/news/seed-siggraph2017-compute-for-graphics https://www.ea.com/seed/news/seed-siggraph2017-compute-for-g... DX etc have been slowly exposing some of the underlying SIMD but it is still not really on the level of what is available in a full SIMD model like we have with AVX etc. GPU languages like HLSL/GLSL are designed for ease of use and leave some performance on the ground.
- pjmlp 6y ago
  The path forward seems to be CUDA style programming, it is what Optix does. However I would rather prefer some kind of "SQL" for GPU programming.

Why are Intel GPUs designed in such a way that typical GPU languages don’t fully exploit it? Is the new Xe architecture still SIMD?

by skavi 6y ago

Why are Intel GPUs designed in such a way that typical GPU languages don’t fully exploit it? Is the new Xe architecture still SIMD?

pauljurczak 6y ago

The Intel Gen GPU architecture, which includes the newest incarnation Gen12 aka Xe, is a SIMD architecture as opposed to Nvidia and AMD SIMT architecture. The reasons are historical, i.e. CPU centric design: x86 CPU, Larrabee, Xeon-Phi, etc.

dragontamer 6y ago

OpenCL works on Intel GPUs, while CUDA doesn't because CUDA is an NVidia technology. > Is the new Xe architecture still SIMD? SIMD is... pretty much all GPUs do. There's a few scalar bits here and there to speed up if-statements and the like, but the entire point of a GPU is to build a machine for SIMD.

oivey 6y ago
OpenCL is basically dead at this point, too. The de facto standard is CUDA and there aren’t currently any real challengers. Maybe eventually AMD’s ROCm or Intel’s oneAPI will get traction.
- pjmlp 6y ago
  For them to get traction, they need to invest in debugger tooling that allows the productivity as on CPUs, and to help language communities other than C and C++ to target GPGPUs. NVidia started doing both around CUDA 3.0, whereas Khronos, AMD and Intel only started paying attention that not everyone wanted to do printf() style debugging with a C dialect until it was too late to get people's attention back.
  - MaxBarraclough 6y ago
    AMD had a good Visual Studio plugin for OpenCL, complete with debugging support, although I believe it's since been discontinued.
- profquail 6y ago
  oneAPI uses DPC++ (Data-Parallel C++), which is pretty much just SYCL, which itself is a C++ library on top of OpenCL. From my understanding, the Khronos group realized OpenCL 2.x was much too complicated so vendors just weren’t implementing it, or only implementing parts of it, so they came up with OpenCL 3.0 which is slimmed-down and much more modular. It’s hard to say how much adoption it’ll get, but with Intel focused on DPC++ and oneAPI now, there will definitely be more numerical software coming out in the next few years that compiles down to and runs on OpenCL. For example, Intel engineers are building a numpy clone on top of DPC++, so unlike regular numpy it’ll take advantage of multiple CPU cores: https://github.com/IntelPython/dpnp https://github.com/IntelPython/dpnp
  - astrange 6y ago
    > From my understanding, the Khronos group realized OpenCL 2.x was much too complicated so vendors just weren’t implementing it, or only implementing parts of it, so they came up with OpenCL 3.0 which is slimmed-down and much more modular. Something like this also happened to OpenGL 4.3. It added a compute shader extension which was essentially all of OpenCL again, except different, so you had 2x the implementation work. This is about when some people stopped implementing OpenGL.
    - TazeTSchnitzel 6y ago
      OpenGL compute shaders are a natural step if you have unified programmable shader cores, and less complicated than adding new pipeline stages for everything (tessellation shaders, geometry shaders, …). Khronos could have chosen only to add OpenCL integration, but OpenCL C is a very different language to GLSL, the memory model (among other things) is different, and so on. I don't see why video game developers should be forced to use OpenCL when they want to work with the outputs of OpenGL rendering passes, to produce inputs to OpenGL rendering passes, scheduled in OpenGL, to do things that don't fit neatly into vertex or fragment shaders?
    - pjmlp 6y ago
      On the contrary, for example on Android OpenGL ES you get compute shaders, but no OpenCL. By the way, the latest version is OpenGL 4.6, it is also available on the Switch.
- gspr 6y ago
  I realize the size of the ecosystem is important, but if you're writing all the GPU code yourself, I wouldn't say OpenCL is dead. I recently had a chance to learn the basics for a work project, never having touched the field before. I picked OpenCL, because I knew I was writing all my non-BLAS code myself, and there's no way in hell I'll voluntarily lock myself into a closed ecosystem. (PS: CLBlast, which is different from CLblas, is a joy!) I was pleasantly surprised. I found OpenCL very nice to work with indeed! And my code runs on any modern GPU out there. I've tested it on Intel integrated GPUs, AMD GPUs, and Nvidia's fancy datacenter devices. And even CPUs. Seamlessly, through a runtime switch fully controlled by the application itself! Now, could I have gotten more performance out of CUDA? Yeah, I estimate about a factor 2. For the cost of tying myself to a proprietary, locked in technology from a hostile vendor, throwing out two major classes of devices, and losing the ability to test out code anywhere. Not worth it. I hope OpenCL has life in it still. The stuff I keep reading that CUDA is far easier to approach definitely did not ring true to this beginner.
astrange 6y ago
GPUs don't need to have SIMD instructions; if you give one a fully scalar program it just needs to run a lot of copies of it at once. Every architecture is different here, including within the same vendor.
- dragontamer 6y ago
  > GPUs don't need to have SIMD instructions; Except NVidia Ampere (RTX 3xxx series) and AMD RDNA2 (Navi / 6xxx series) are both SIMD architectures with SIMD-instructions. And the #3 company: Intel, also has SIMD instructions. I know that some GPUs out there are VLIW or other weird architectures, but... the "big 3" are SIMD-based. > if you give one a fully scalar program it just needs to run a lot of copies of it at once. Its emulated on a SIMD processor. That SIMD processor will suffer branch-divergence as you traverse through if-statements and while-statements, because its physically SIMD. The compiler / programming model is scalar. But the assembly instructions are themselves vector. Yeah, NVidia now has per-SIMD core instruction pointers. But that doesn't mean that the hardware can physically execute different instructions: they're still all locked together with SIMD-style at the physical level.
  - my123 6y ago
    > with SIMD-instructions You assume so, for Volta onwards that's not true. Each SIMT thread on NVIDIA GPUs for Volta onwards has its own instruction pointer. And yes, it's scalar instructions at the ISA level.
    - dragontamer 6y ago
      > You assume so, for Volta onwards that's not true. https://arxiv.org/pdf/1804.06826.pdf https://arxiv.org/pdf/1804.06826.pdf NVidia's SASS on Volta is pretty clearly SIMD-instructions. The PTX "virtual machine assembly" is fully documented. SASS is not documented, but it ain't a secret either. SASS follows closely with PTX, with exception of some "barrier bits" that seem to track dependencies (probably compiler-managed read/write dependency chains). > Each SIMT thread on NVIDIA GPUs for Volta onwards has its own instruction pointer. And yes, it's scalar instructions at the ISA level. When your "scalar" instruction executes 32-threads in parallel (subject to an execution mask), that's called SIMD. The instruction pointer is to resolve deadlock conditions with locks, and the 32-wide SIMD cores will execute one-at-a-time to prevent deadlocks. But your goal as a GPU programmer is to get as much 32-wide execution going on as possible. 1-at-a-time serialization is very, very bad for performance. Its POSSIBLE to do on Volta, but highly recommended you stay away from that cornercase (you lose over 95% of your performance).

TinkersW 6y ago

The paper covers the reasons, and they aren't new or specific to Intel. There was a talk by Andrew Lauritzen basically saying the same thing a few years back. https://www.ea.com/seed/news/seed-siggraph2017-compute-for-graphics https://www.ea.com/seed/news/seed-siggraph2017-compute-for-g... DX etc have been slowly exposing some of the underlying SIMD but it is still not really on the level of what is available in a full SIMD model like we have with AVX etc. GPU languages like HLSL/GLSL are designed for ease of use and leave some performance on the ground.

pjmlp 6y ago
The path forward seems to be CUDA style programming, it is what Optix does. However I would rather prefer some kind of "SQL" for GPU programming.