5 ms·

how to do everything in parallel I found it was worse than that when I started playing with shaders, because my idea of parallelism came from threads, workers,

how to do everything in parallel I found it was worse than that when I started playing with shaders, because my idea of parallelism came from threads, workers, and async functions eg doing several separate tasks at the same time. Parallel on a GPU is more like running lots of the same function each with different parameters at the same time eg running a function to calculate the color of a pixel using its x and y coordinates (and maybe some other uniforms) for every pixel on the screen.

delta_p_delta_x 3y ago
I like to put it like this: a GPU is a really, really fast Map-Reduce hardware processor. You give it a bunch of data with elements which are all independent of each other, apply the same transformations to to each element, and output a single data structure (a colour attachment or render target, or some compute buffer). This explains a lot of the shortcomings and design decisions of GPUs. They prioritise throughput over latency; they have a lot of memory but much slower than the cores themselves (~500 – 700 cycle latency compared to ~20 – 40 cycle latency on the CPU); they run 'threads' in synchronised lock-step, called warps; branching on a GPU is particularly expensive—if one shader for one fragment needs to branch, then the entire warp that it's on will have to wait for it, or worse still, calculate that branch, too; Their 3D graphics legacy still endures: despite the advent of GPGPU, they still have plenty of dedicated hardware for the graphics pipeline, including rasterisers, texture-mapping units, render output processors, etc. Once people get this idea, then GPU programming is easier to reason about. One execution of a shader deals with one fragment (actually four, but at a high level of abstraction, this is sufficient). Oddly enough, I did GPU parallelisation first, and then came to CPU parallelisation. I find CPU-level parallelism and concurrency problems like OS inter-process synchronisation, async-await, etc. much harder to reason about, because they generally cannot be reduced to such a simplistic 'one thread for one pixel'. It's also why I like embarrassingly parallel problems, because they're the most easily GPU-able.
- sspiff 3y ago
  A GPU is a really, really fast map hardware processor. Reduce steps are not good at utilizing all of the silicon hardware available IIRC.
  - KeplerBoy 3y ago
    true, for reductions you have algorithms like prefix scans, which produce a lot of output you're not necessarily interested in or algorithms which utilize fewer and fewer parts of the chips as the reduction progresses.
raincole 3y ago
I remember how shock I was when the first time I leanred branching (arguably the most basic building block of programming) can severely affect GPU's performance. *techinically it's even true for CPU in some cases...
- c4mpute 3y ago
  > *techinically it's even true for CPU in some cases... Practically in all cases. It isn't as bad to branch as it is in a GPU. But branch prediction in CPUs is an important factor for performance critical code, and getting it right (e.g. by switching the 'branch/no branch' conditions in assembler or using branch hints such as likely/unlikely in the linux kernel) can make a very noticable difference. I've seen a factor of 2 in some tight loops, and 10% should often be achievable. And sometimes, by using bitmasks or multiplication by 1/0, one can design branchless code which can enable the use of e.g. more AVX instructions on CPUs, while also avoiding branch miss penalties.
- amagine 3y ago
  Branching isn't in and of itself too bad. It's divergent branching within a warp, especially if the blocks are large, that can kill perf.

delta_p_delta_x 3y ago

I like to put it like this: a GPU is a really, really fast Map-Reduce hardware processor. You give it a bunch of data with elements which are all independent of each other, apply the same transformations to to each element, and output a single data structure (a colour attachment or render target, or some compute buffer). This explains a lot of the shortcomings and design decisions of GPUs. They prioritise throughput over latency; they have a lot of memory but much slower than the cores themselves (~500 – 700 cycle latency compared to ~20 – 40 cycle latency on the CPU); they run 'threads' in synchronised lock-step, called warps; branching on a GPU is particularly expensive—if one shader for one fragment needs to branch, then the entire warp that it's on will have to wait for it, or worse still, calculate that branch, too; Their 3D graphics legacy still endures: despite the advent of GPGPU, they still have plenty of dedicated hardware for the graphics pipeline, including rasterisers, texture-mapping units, render output processors, etc. Once people get this idea, then GPU programming is easier to reason about. One execution of a shader deals with one fragment (actually four, but at a high level of abstraction, this is sufficient). Oddly enough, I did GPU parallelisation first, and then came to CPU parallelisation. I find CPU-level parallelism and concurrency problems like OS inter-process synchronisation, async-await, etc. much harder to reason about, because they generally cannot be reduced to such a simplistic 'one thread for one pixel'. It's also why I like embarrassingly parallel problems, because they're the most easily GPU-able.

sspiff 3y ago
A GPU is a really, really fast map hardware processor. Reduce steps are not good at utilizing all of the silicon hardware available IIRC.
- KeplerBoy 3y ago
  true, for reductions you have algorithms like prefix scans, which produce a lot of output you're not necessarily interested in or algorithms which utilize fewer and fewer parts of the chips as the reduction progresses.

raincole 3y ago

I remember how shock I was when the first time I leanred branching (arguably the most basic building block of programming) can severely affect GPU's performance. *techinically it's even true for CPU in some cases...

c4mpute 3y ago
> *techinically it's even true for CPU in some cases... Practically in all cases. It isn't as bad to branch as it is in a GPU. But branch prediction in CPUs is an important factor for performance critical code, and getting it right (e.g. by switching the 'branch/no branch' conditions in assembler or using branch hints such as likely/unlikely in the linux kernel) can make a very noticable difference. I've seen a factor of 2 in some tight loops, and 10% should often be achievable. And sometimes, by using bitmasks or multiplication by 1/0, one can design branchless code which can enable the use of e.g. more AVX instructions on CPUs, while also avoiding branch miss penalties.
amagine 3y ago
Branching isn't in and of itself too bad. It's divergent branching within a warp, especially if the blocks are large, that can kill perf.