3 ms·

MIMD Interpretation on a GPU [pdf]

27 points by ColinWright 4y ago

dragontamer 4y ago
So first off, GPUs are MIMD, in that each compute-unit / symmetric multiprocessor is independent of one another. NVidia's 3xxx line of GPUs are further 2-instructions per clock tick / superscalar, which is more "traditional" MIMD, in that the cores are now doing 2 different things per clock tick on top of all the SIMD-goodness they're doing. AMD is mostly SIMD, but arguably MIMD from the perspective that it could be executing a "scalar" instruction (branch / loop instructions) in parallel with a "vector" instruction (add/subtract/multiply) ---------- So we can say that the assembly-language model of NVidia PTX / AMD RDNA/CDNA is SIMD, but the execution is now MIMD. This pdf would then add a virtual machine on top of that, giving us an MIMD virtual-machine that compiles into SIMD instructions that's executed on MIMD hardware. I dunno where I'm going with this, I'm just amused by it, that's all. Each step makes sense but its kinda funny when you look at it all from the top down.
- moonchild 4y ago
  Of course GPUs are MIMD. They are also SIMD (or SIMT if you prefer). The same is true for CPUs. Parallelism is balanced along multiple axes, and if your problem is sufficiently heterogeneous, it seems completely plausible that the ideas in the linked paper would be useful for extracting additional parallelism from the hardware.
- sitkack 4y ago
  > So first off, GPUs are MIMD, in that each compute-unit / symmetric multiprocessor is independent of one another. This paper was from a SuperComputer 09, pretty sure the researchers are aware. Your comment would have great without the first line.
- lostmsu 4y ago
  Where can I read more on MIMD in 30xx?
  - dragontamer 4y ago
    https://www.nvidia.com/content/PDF/nvidia-ampere-ga-102-gpu-architecture-whitepaper-v2.1.pdf https://www.nvidia.com/content/PDF/nvidia-ampere-ga-102-gpu-... Page 12. > In the Turing generation, each of the four SM processing blocks (also called partitions) had two primary datapaths, but only one of the two could process FP32 operations. The other datapath was limited to integer operations. GA10X includes FP32 processing on both datapaths, doubling the peak processing rate for FP32 operations. One datapath in each partition consists of 16 FP32 CUDA Cores capable of executing 16 FP32 operations per clock. Another datapath consists of both 16 FP32 CUDA Cores and 16 INT32 Cores, and is capable of executing either 16 FP32 operations OR 16 INT32 operations per clock. As a result of this new design, each GA10x SM partition is capable of executing either 32 FP32 operations per clock, or 16 FP32 and 16 INT32 operations per clock. All four SM partitions combined can execute 128 FP32 operations per clock, which is double the FP32 rate of the Turing SM, or 64 FP32 and 64 INT32 operations per clock.
    - lostmsu 4y ago
      As a non-involved, not sure I understand this quote correctly. Reads like Turing had 4 independent core equivalents, but 2 of them were incapable of FP; and Ampere has all 4 cores capable of doing either. But how are shaders scheduled to those cores? As a programmer, how do I see those cores and send code to run on them? Are there basic samples?
      - dragontamer 4y ago
        An NVidia 'SM' is the closest thing to a CPU core. Each SM has multiple MIMD capabilities as it is, but I was focusing upon the dual-issue pipelines of FP32+integer instructions simultaneously. > As a programmer, how do I see those cores and send code to run on them? When the following assembly instructions are send to the GPU, they will execute simultaneously add.s32 r1, r2 add.f32 r3, r4 That is: the NVidia SM can process a floating point (f32) instruction simultaneously with integer (s32) instructions. ------ It sounds like you are getting confused by a lot of other features in the GPU SM. I dunno where exactly, but it's a complicated beast. All I really think I can say is to study up on GPU microarchitecture for more details, there is a lot going on here and it'd be too difficult for me to cover it all in just a few comments.
HappySweeney 4y ago
> The toolchain is expected to become a public domain source code release as soon as it has reached an appropriate level of usability and stability. So, never?
- PAPPPmAc 4y ago
  There's a public copy of the (super sketchy) draft toolchain that some of the folks in the lab were working with on github: https://github.com/aggregate/MOG https://github.com/aggregate/MOG It's a horrifying toolchain that uses MIPSEL assembly as an IR so we didn't have to screw around with language front-ends, but it pretty much worked.
  - sitkack 4y ago
    > MIPSEL assembly as an IR http://nestedvm.ibex.org/ http://nestedvm.ibex.org/ > NestedVM provides binary translation for Java Bytecode. This is done by having GCC compile to a MIPS binary which is then translated to a Java class file. Funny, it seems they are both from the same time period. ---- > Our first MIMD interpreter running on the MasPar MP1 [6] achieved approximately 1/4 the theoretical peak native distributed-memory SIMD speed while supporting a full-featured shared-memory MIMD programming model. That is amazing! If you haven't seen the project below, you should definitely check it out. Linux in a Pixel Shader - A RISC-V Emulator for VRChat https://blog.pimaker.at/texts/rvc1/ https://blog.pimaker.at/texts/rvc1/ No one seems to have taken up this line of research, most likely because it was too outside the norm, yet the results are amazing. https://www.semanticscholar.org/paper/MIMD-Interpretation-on-a-GPU-Dietz-Young/f67138d4b64ebd9ecda6ccd7fa3378b62f051cf4?sort=relevance&citationIntent=background https://www.semanticscholar.org/paper/MIMD-Interpretation-on...

27 points by ColinWright 4y ago

So first off, GPUs are MIMD, in that each compute-unit / symmetric multiprocessor is independent of one another. NVidia's 3xxx line of GPUs are further 2-instructions per clock tick / superscalar, which is more "traditional" MIMD, in that the cores are now doing 2 different things per clock tick on top of all the SIMD-goodness they're doing. AMD is mostly SIMD, but arguably MIMD from the perspective that it could be executing a "scalar" instruction (branch / loop instructions) in parallel with a "vector" instruction (add/subtract/multiply) ---------- So we can say that the assembly-language model of NVidia PTX / AMD RDNA/CDNA is SIMD, but the execution is now MIMD. This pdf would then add a virtual machine on top of that, giving us an MIMD virtual-machine that compiles into SIMD instructions that's executed on MIMD hardware. I dunno where I'm going with this, I'm just amused by it, that's all. Each step makes sense but its kinda funny when you look at it all from the top down.

moonchild 4y ago
Of course GPUs are MIMD. They are also SIMD (or SIMT if you prefer). The same is true for CPUs. Parallelism is balanced along multiple axes, and if your problem is sufficiently heterogeneous, it seems completely plausible that the ideas in the linked paper would be useful for extracting additional parallelism from the hardware.
sitkack 4y ago
> So first off, GPUs are MIMD, in that each compute-unit / symmetric multiprocessor is independent of one another. This paper was from a SuperComputer 09, pretty sure the researchers are aware. Your comment would have great without the first line.
lostmsu 4y ago
Where can I read more on MIMD in 30xx?
- dragontamer 4y ago
  https://www.nvidia.com/content/PDF/nvidia-ampere-ga-102-gpu-architecture-whitepaper-v2.1.pdf https://www.nvidia.com/content/PDF/nvidia-ampere-ga-102-gpu-... Page 12. > In the Turing generation, each of the four SM processing blocks (also called partitions) had two primary datapaths, but only one of the two could process FP32 operations. The other datapath was limited to integer operations. GA10X includes FP32 processing on both datapaths, doubling the peak processing rate for FP32 operations. One datapath in each partition consists of 16 FP32 CUDA Cores capable of executing 16 FP32 operations per clock. Another datapath consists of both 16 FP32 CUDA Cores and 16 INT32 Cores, and is capable of executing either 16 FP32 operations OR 16 INT32 operations per clock. As a result of this new design, each GA10x SM partition is capable of executing either 32 FP32 operations per clock, or 16 FP32 and 16 INT32 operations per clock. All four SM partitions combined can execute 128 FP32 operations per clock, which is double the FP32 rate of the Turing SM, or 64 FP32 and 64 INT32 operations per clock.
  - lostmsu 4y ago
    As a non-involved, not sure I understand this quote correctly. Reads like Turing had 4 independent core equivalents, but 2 of them were incapable of FP; and Ampere has all 4 cores capable of doing either. But how are shaders scheduled to those cores? As a programmer, how do I see those cores and send code to run on them? Are there basic samples?
    - dragontamer 4y ago
      An NVidia 'SM' is the closest thing to a CPU core. Each SM has multiple MIMD capabilities as it is, but I was focusing upon the dual-issue pipelines of FP32+integer instructions simultaneously. > As a programmer, how do I see those cores and send code to run on them? When the following assembly instructions are send to the GPU, they will execute simultaneously add.s32 r1, r2 add.f32 r3, r4 That is: the NVidia SM can process a floating point (f32) instruction simultaneously with integer (s32) instructions. ------ It sounds like you are getting confused by a lot of other features in the GPU SM. I dunno where exactly, but it's a complicated beast. All I really think I can say is to study up on GPU microarchitecture for more details, there is a lot going on here and it'd be too difficult for me to cover it all in just a few comments.

HappySweeney 4y ago

> The toolchain is expected to become a public domain source code release as soon as it has reached an appropriate level of usability and stability. So, never?

PAPPPmAc 4y ago
There's a public copy of the (super sketchy) draft toolchain that some of the folks in the lab were working with on github: https://github.com/aggregate/MOG https://github.com/aggregate/MOG It's a horrifying toolchain that uses MIPSEL assembly as an IR so we didn't have to screw around with language front-ends, but it pretty much worked.
- sitkack 4y ago
  > MIPSEL assembly as an IR http://nestedvm.ibex.org/ http://nestedvm.ibex.org/ > NestedVM provides binary translation for Java Bytecode. This is done by having GCC compile to a MIPS binary which is then translated to a Java class file. Funny, it seems they are both from the same time period. ---- > Our first MIMD interpreter running on the MasPar MP1 [6] achieved approximately 1/4 the theoretical peak native distributed-memory SIMD speed while supporting a full-featured shared-memory MIMD programming model. That is amazing! If you haven't seen the project below, you should definitely check it out. Linux in a Pixel Shader - A RISC-V Emulator for VRChat https://blog.pimaker.at/texts/rvc1/ https://blog.pimaker.at/texts/rvc1/ No one seems to have taken up this line of research, most likely because it was too outside the norm, yet the results are amazing. https://www.semanticscholar.org/paper/MIMD-Interpretation-on-a-GPU-Dietz-Young/f67138d4b64ebd9ecda6ccd7fa3378b62f051cf4?sort=relevance&citationIntent=background https://www.semanticscholar.org/paper/MIMD-Interpretation-on...