3 ms·

How do you get started with learning GPU programming? I mean that in a pedagogical way, like what is the best educational way forward. Build stupid stuff in sha

by wrnr 6y ago

How do you get started with learning GPU programming? I mean that in a pedagogical way, like what is the best educational way forward. Build stupid stuff in shader toys?

raphlinus 6y ago
It's a good question, and it's been suggested I write a GPU compute 101. I did do a talk at Jane Street earlier in the year called "A taste of GPU compute" which has some relevant resources (a Google search on that term will give you a number of links). I think there are probably a number of on-ramps. One easy way to get started is shadertoy, which if nothing else should give familiarity with GLSL syntax and intuition for performance in the simplest case: each "thread" computes something (a pixel value) independently of the others. You'll quickly run into limitations though, as it can't really do any of the more advanced compute stuff. I think as WebGPU becomes real, an analogous tool that unlocks compute kernels (aka compute shaders) could be very powerful for pedagogy. I think most people that do compute on GPU use CUDA, as it's the only really practical toolchain for it. That has a large number of high quality open source libraries, which tend to be well-documented and have good research papers behind them. You can start by using these libraries, then digging deeper to see how they're implemented. As I've been going on about, I believe this space is ripe for major growth in the next decade or so. As a rough guide, if you can make your algorithm fit the GPU compute model nicely, you'll get about 10x the compute per dollar, which is effectively the same as compute per watt. Why leave such performance on the table? The answer is that programming GPUs is just too hard. In certain areas, including machine learning, an investment of research has partially fixed that problem. But in others there is fruit at medium height, ripe for the picking. And in some other areas, you'll need a tall ladder, of which I believe a solid understanding of monoids is but one rung.
- wrnr 6y ago
  Thanks, it was your work that kindled my interest in programming the GPU. The fact that most vector graphics are done by CPU libraries made me resize it's still a very underdeveloped area. The hard thing about it is also what makes it interesting, it's not just a different programming language but a different architecture with other runtime characteristics. Sort of the closest you can get to owning a quantum computer. Anyway I'll learn some math and cuda, and buy the book if you find the time :)
Jhsto 6y ago
In a sense GPU programming is similar to cellular automata. Each cell is an invocation on the GPU, and you can assume that you may have unlimited number of these invocations. You can freeze cells infinitely or temporarily, but you cannot group them. You can assume you know the identifier number of each cell, and you can assume you know the number of cells for each program. The hard part is now to generalize algorithms in which only control structures are based on freezing, identifying, and counting of cells. The cells can communicate information to the neighboring cells, and to any arbitrary cell, but the latter is not good for performance. Most universities don't really teach parallel algorithm development in any way despite the core count increasing in both CPU and GPU world. Current languages like GLSL and CUDA are very abstracted in the sense that they don't really tell why some approach is better or worse than the other. To me, if you can create as general cellular automata -like solution, then such logic can be converted and is generally fine performance-wise for SIMD orientated programming.
gitgud 6y ago
Shader toys are definitely a fun way to get your head around the GPU, reading and forking other people's shaders is great to learn. I made this [1] simple shader because I couldn't find the effect in Open-Source video software... [1] https://www.shadertoy.com/view/WtfGzX https://www.shadertoy.com/view/WtfGzX

by wrnr 6y ago

How do you get started with learning GPU programming? I mean that in a pedagogical way, like what is the best educational way forward. Build stupid stuff in shader toys?

raphlinus 6y ago

It's a good question, and it's been suggested I write a GPU compute 101. I did do a talk at Jane Street earlier in the year called "A taste of GPU compute" which has some relevant resources (a Google search on that term will give you a number of links). I think there are probably a number of on-ramps. One easy way to get started is shadertoy, which if nothing else should give familiarity with GLSL syntax and intuition for performance in the simplest case: each "thread" computes something (a pixel value) independently of the others. You'll quickly run into limitations though, as it can't really do any of the more advanced compute stuff. I think as WebGPU becomes real, an analogous tool that unlocks compute kernels (aka compute shaders) could be very powerful for pedagogy. I think most people that do compute on GPU use CUDA, as it's the only really practical toolchain for it. That has a large number of high quality open source libraries, which tend to be well-documented and have good research papers behind them. You can start by using these libraries, then digging deeper to see how they're implemented. As I've been going on about, I believe this space is ripe for major growth in the next decade or so. As a rough guide, if you can make your algorithm fit the GPU compute model nicely, you'll get about 10x the compute per dollar, which is effectively the same as compute per watt. Why leave such performance on the table? The answer is that programming GPUs is just too hard. In certain areas, including machine learning, an investment of research has partially fixed that problem. But in others there is fruit at medium height, ripe for the picking. And in some other areas, you'll need a tall ladder, of which I believe a solid understanding of monoids is but one rung.

wrnr 6y ago
Thanks, it was your work that kindled my interest in programming the GPU. The fact that most vector graphics are done by CPU libraries made me resize it's still a very underdeveloped area. The hard thing about it is also what makes it interesting, it's not just a different programming language but a different architecture with other runtime characteristics. Sort of the closest you can get to owning a quantum computer. Anyway I'll learn some math and cuda, and buy the book if you find the time :)

Jhsto 6y ago

In a sense GPU programming is similar to cellular automata. Each cell is an invocation on the GPU, and you can assume that you may have unlimited number of these invocations. You can freeze cells infinitely or temporarily, but you cannot group them. You can assume you know the identifier number of each cell, and you can assume you know the number of cells for each program. The hard part is now to generalize algorithms in which only control structures are based on freezing, identifying, and counting of cells. The cells can communicate information to the neighboring cells, and to any arbitrary cell, but the latter is not good for performance. Most universities don't really teach parallel algorithm development in any way despite the core count increasing in both CPU and GPU world. Current languages like GLSL and CUDA are very abstracted in the sense that they don't really tell why some approach is better or worse than the other. To me, if you can create as general cellular automata -like solution, then such logic can be converted and is generally fine performance-wise for SIMD orientated programming.

gitgud 6y ago

Shader toys are definitely a fun way to get your head around the GPU, reading and forking other people's shaders is great to learn. I made this [1] simple shader because I couldn't find the effect in Open-Source video software... [1] https://www.shadertoy.com/view/WtfGzX https://www.shadertoy.com/view/WtfGzX