It's a good question, and it's been suggested I write a GPU compute 101. I did do a talk at Jane Street earlier in the year called "A taste of GPU compute" whic

It's a good question, and it's been suggested I write a GPU compute 101. I did do a talk at Jane Street earlier in the year called "A taste of GPU compute" which has some relevant resources (a Google search on that term will give you a number of links). I think there are probably a number of on-ramps. One easy way to get started is shadertoy, which if nothing else should give familiarity with GLSL syntax and intuition for performance in the simplest case: each "thread" computes something (a pixel value) independently of the others. You'll quickly run into limitations though, as it can't really do any of the more advanced compute stuff. I think as WebGPU becomes real, an analogous tool that unlocks compute kernels (aka compute shaders) could be very powerful for pedagogy. I think most people that do compute on GPU use CUDA, as it's the only really practical toolchain for it. That has a large number of high quality open source libraries, which tend to be well-documented and have good research papers behind them. You can start by using these libraries, then digging deeper to see how they're implemented. As I've been going on about, I believe this space is ripe for major growth in the next decade or so. As a rough guide, if you can make your algorithm fit the GPU compute model nicely, you'll get about 10x the compute per dollar, which is effectively the same as compute per watt. Why leave such performance on the table? The answer is that programming GPUs is just too hard. In certain areas, including machine learning, an investment of research has partially fixed that problem. But in others there is fruit at medium height, ripe for the picking. And in some other areas, you'll need a tall ladder, of which I believe a solid understanding of monoids is but one rung.