4 ms·

Interesting, but very heavy read. And this is about the low level access towards it via system libaries. A more simple way to get into GPU programming, would b

Interesting, but very heavy read. And this is about the low level access towards it via system libaries. A more simple way to get into GPU programming, would be WebGPU. I can recommend this tutorial as an introduction: https://surma.dev/things/webgpu/ https://surma.dev/things/webgpu/ GPU programming is really a different way of programming, even though the code looks mostly familiar. And if you are used to single threaded, it will give you a massive headache thinking about, how to do everything in parallel, so that everything is not blocking everything. The raw power is tempting, so much that I evaluate now how to do allmost everything on the GPU, but many things simply are not suitable to do there. Basically, you want to avoid conditions and loops. Cannot do recursion. Debugging is a pain and non obvious things can make everything very slow, until you roughly understand, what is going on inside.

onion2k 3y ago
how to do everything in parallel I found it was worse than that when I started playing with shaders, because my idea of parallelism came from threads, workers, and async functions eg doing several separate tasks at the same time. Parallel on a GPU is more like running lots of the same function each with different parameters at the same time eg running a function to calculate the color of a pixel using its x and y coordinates (and maybe some other uniforms) for every pixel on the screen.
- delta_p_delta_x 3y ago
  I like to put it like this: a GPU is a really, really fast Map-Reduce hardware processor. You give it a bunch of data with elements which are all independent of each other, apply the same transformations to to each element, and output a single data structure (a colour attachment or render target, or some compute buffer). This explains a lot of the shortcomings and design decisions of GPUs. They prioritise throughput over latency; they have a lot of memory but much slower than the cores themselves (~500 – 700 cycle latency compared to ~20 – 40 cycle latency on the CPU); they run 'threads' in synchronised lock-step, called warps; branching on a GPU is particularly expensive—if one shader for one fragment needs to branch, then the entire warp that it's on will have to wait for it, or worse still, calculate that branch, too; Their 3D graphics legacy still endures: despite the advent of GPGPU, they still have plenty of dedicated hardware for the graphics pipeline, including rasterisers, texture-mapping units, render output processors, etc. Once people get this idea, then GPU programming is easier to reason about. One execution of a shader deals with one fragment (actually four, but at a high level of abstraction, this is sufficient). Oddly enough, I did GPU parallelisation first, and then came to CPU parallelisation. I find CPU-level parallelism and concurrency problems like OS inter-process synchronisation, async-await, etc. much harder to reason about, because they generally cannot be reduced to such a simplistic 'one thread for one pixel'. It's also why I like embarrassingly parallel problems, because they're the most easily GPU-able.
  - sspiff 3y ago
    A GPU is a really, really fast map hardware processor. Reduce steps are not good at utilizing all of the silicon hardware available IIRC.
    - KeplerBoy 3y ago
      true, for reductions you have algorithms like prefix scans, which produce a lot of output you're not necessarily interested in or algorithms which utilize fewer and fewer parts of the chips as the reduction progresses.
- raincole 3y ago
  I remember how shock I was when the first time I leanred branching (arguably the most basic building block of programming) can severely affect GPU's performance. *techinically it's even true for CPU in some cases...
  - c4mpute 3y ago
    > *techinically it's even true for CPU in some cases... Practically in all cases. It isn't as bad to branch as it is in a GPU. But branch prediction in CPUs is an important factor for performance critical code, and getting it right (e.g. by switching the 'branch/no branch' conditions in assembler or using branch hints such as likely/unlikely in the linux kernel) can make a very noticable difference. I've seen a factor of 2 in some tight loops, and 10% should often be achievable. And sometimes, by using bitmasks or multiplication by 1/0, one can design branchless code which can enable the use of e.g. more AVX instructions on CPUs, while also avoiding branch miss penalties.
  - amagine 3y ago
    Branching isn't in and of itself too bad. It's divergent branching within a warp, especially if the blocks are large, that can kill perf.
nromiun 3y ago
Not really, WebGPU API is around the same level as CUDA or OpenCL. In fact SYCL (which is used in this article) is slightly higher than WebGPU, because you don't have to manually allocate buffers. If you want something even higher you should look into frameworks like Numba (for CUDA) and Taichi (for Vulkan and WebGPU).
- hutzlibu 3y ago
  But WebGPU you can just try out directly in the browser and immediately start hacking away (if you have chrome, or activate some flags in firefox): https://google.github.io/tour-of-wgsl/ https://google.github.io/tour-of-wgsl/ No need to setup, install or config anything. I think that is arguably easier as a quick start. Otherwise yes, the whole tooling and everything else is not really that mature in WebGPU. And thanks for Taichi. I did not know of that libary and it sounds interesting! (I will stay with WebGPU for now)
  - nromiun 3y ago
    Easier, sure, but not higher level compared to CUDA or OpenCL. You can also (probably) use Taichi.js without installing anything. Which is a high level API.
    - hutzlibu 3y ago
      "You can also (probably) use Taichi.js without installing anything" Yes you can! (because of WebGPU) And I never said WebGPU is high level. I said and meant it is simpler to get started with. At least it was quite simple for me. But WebGPU and wgsl by itself are surely not "simple".
6keZbCECT2uB 3y ago
It is a pretty different way of programming, and that's part of what makes it so fun. Within constraints, you get a really interactive way to design algorithms which are much more parallel than what is feasible to write for a CPU. Debugging is so much better these days with some support for debuggers and printf (at least on CUDA). Maybe the same facilities aren't available for WebGPU? Conditions and loops are largely fine as long as you avoid warp divergence and such.

onion2k 3y ago

how to do everything in parallel I found it was worse than that when I started playing with shaders, because my idea of parallelism came from threads, workers, and async functions eg doing several separate tasks at the same time. Parallel on a GPU is more like running lots of the same function each with different parameters at the same time eg running a function to calculate the color of a pixel using its x and y coordinates (and maybe some other uniforms) for every pixel on the screen.

delta_p_delta_x 3y ago
I like to put it like this: a GPU is a really, really fast Map-Reduce hardware processor. You give it a bunch of data with elements which are all independent of each other, apply the same transformations to to each element, and output a single data structure (a colour attachment or render target, or some compute buffer). This explains a lot of the shortcomings and design decisions of GPUs. They prioritise throughput over latency; they have a lot of memory but much slower than the cores themselves (~500 – 700 cycle latency compared to ~20 – 40 cycle latency on the CPU); they run 'threads' in synchronised lock-step, called warps; branching on a GPU is particularly expensive—if one shader for one fragment needs to branch, then the entire warp that it's on will have to wait for it, or worse still, calculate that branch, too; Their 3D graphics legacy still endures: despite the advent of GPGPU, they still have plenty of dedicated hardware for the graphics pipeline, including rasterisers, texture-mapping units, render output processors, etc. Once people get this idea, then GPU programming is easier to reason about. One execution of a shader deals with one fragment (actually four, but at a high level of abstraction, this is sufficient). Oddly enough, I did GPU parallelisation first, and then came to CPU parallelisation. I find CPU-level parallelism and concurrency problems like OS inter-process synchronisation, async-await, etc. much harder to reason about, because they generally cannot be reduced to such a simplistic 'one thread for one pixel'. It's also why I like embarrassingly parallel problems, because they're the most easily GPU-able.
- sspiff 3y ago
  A GPU is a really, really fast map hardware processor. Reduce steps are not good at utilizing all of the silicon hardware available IIRC.
  - KeplerBoy 3y ago
    true, for reductions you have algorithms like prefix scans, which produce a lot of output you're not necessarily interested in or algorithms which utilize fewer and fewer parts of the chips as the reduction progresses.
raincole 3y ago
I remember how shock I was when the first time I leanred branching (arguably the most basic building block of programming) can severely affect GPU's performance. *techinically it's even true for CPU in some cases...
- c4mpute 3y ago
  > *techinically it's even true for CPU in some cases... Practically in all cases. It isn't as bad to branch as it is in a GPU. But branch prediction in CPUs is an important factor for performance critical code, and getting it right (e.g. by switching the 'branch/no branch' conditions in assembler or using branch hints such as likely/unlikely in the linux kernel) can make a very noticable difference. I've seen a factor of 2 in some tight loops, and 10% should often be achievable. And sometimes, by using bitmasks or multiplication by 1/0, one can design branchless code which can enable the use of e.g. more AVX instructions on CPUs, while also avoiding branch miss penalties.
- amagine 3y ago
  Branching isn't in and of itself too bad. It's divergent branching within a warp, especially if the blocks are large, that can kill perf.

nromiun 3y ago

Not really, WebGPU API is around the same level as CUDA or OpenCL. In fact SYCL (which is used in this article) is slightly higher than WebGPU, because you don't have to manually allocate buffers. If you want something even higher you should look into frameworks like Numba (for CUDA) and Taichi (for Vulkan and WebGPU).

hutzlibu 3y ago
But WebGPU you can just try out directly in the browser and immediately start hacking away (if you have chrome, or activate some flags in firefox): https://google.github.io/tour-of-wgsl/ https://google.github.io/tour-of-wgsl/ No need to setup, install or config anything. I think that is arguably easier as a quick start. Otherwise yes, the whole tooling and everything else is not really that mature in WebGPU. And thanks for Taichi. I did not know of that libary and it sounds interesting! (I will stay with WebGPU for now)
- nromiun 3y ago
  Easier, sure, but not higher level compared to CUDA or OpenCL. You can also (probably) use Taichi.js without installing anything. Which is a high level API.
  - hutzlibu 3y ago
    "You can also (probably) use Taichi.js without installing anything" Yes you can! (because of WebGPU) And I never said WebGPU is high level. I said and meant it is simpler to get started with. At least it was quite simple for me. But WebGPU and wgsl by itself are surely not "simple".

6keZbCECT2uB 3y ago

It is a pretty different way of programming, and that's part of what makes it so fun. Within constraints, you get a really interactive way to design algorithms which are much more parallel than what is feasible to write for a CPU. Debugging is so much better these days with some support for debuggers and printf (at least on CUDA). Maybe the same facilities aren't available for WebGPU? Conditions and loops are largely fine as long as you avoid warp divergence and such.