3 ms·

I've been looking into getting into GPU programming, starting with CS334 (https://developer.nvidia.com/udacity-cs344-intro-parallel-programming https://develope

by eachro 3y ago

I've been looking into getting into GPU programming, starting with CS334 (https://developer.nvidia.com/udacity-cs344-intro-parallel-programming https://developer.nvidia.com/udacity-cs344-intro-parallel-pr...) on Udacity. I'm curious to hear from some of the more seasoned GPU veterans out there, what other resources would be good to take a look at after finishing the videos and assignments?

pengaru 3y ago
https://shadertoy.com https://shadertoy.com is a great way to explore shaders
- pjmlp 3y ago
  Indeed, with the caveat that it is constrained to GL ES 3.0 shader capabilities, minus what was removed for WebGL 2.0.
yzh 3y ago
I would recommend the course from Oxford (https://people.maths.ox.ac.uk/gilesm/cuda/ https://people.maths.ox.ac.uk/gilesm/cuda/). Also explore the tutorial section of cutlass (https://github.com/NVIDIA/cutlass/blob/main/media/docs/cute/00_quickstart.md#tutorial https://github.com/NVIDIA/cutlass/blob/main/media/docs/cute/...) if you want to learn more about high performance gemm. OpenAI triton is another good resource if you want to write relatively performant cuda kernels using python for deep learning (https://openai.com/research/triton https://openai.com/research/triton)
gpuhacker 3y ago
If you want to go really in-depth I can recommend GTC on demand. It's Nvidia streaming platform with videos from past GTC conferences. Tony Scuderio had a couple of videos on there called GPU memory bootcamp that are among the best advanced GPU programming learning material out there.
- zetazzed 3y ago
  100% this. You can find all kinds of detailed topics, like CUDA graphs, memory layout optimization, optimizing storage access, etc. https://www.nvidia.com/en-us/on-demand/ https://www.nvidia.com/en-us/on-demand/. They have "playlists" for things like HPC or development tools that collect the most popular videos on those topics.

by eachro 3y ago

pengaru 3y ago

https://shadertoy.com https://shadertoy.com is a great way to explore shaders

pjmlp 3y ago
Indeed, with the caveat that it is constrained to GL ES 3.0 shader capabilities, minus what was removed for WebGL 2.0.

yzh 3y ago

I would recommend the course from Oxford (https://people.maths.ox.ac.uk/gilesm/cuda/ https://people.maths.ox.ac.uk/gilesm/cuda/). Also explore the tutorial section of cutlass (https://github.com/NVIDIA/cutlass/blob/main/media/docs/cute/00_quickstart.md#tutorial https://github.com/NVIDIA/cutlass/blob/main/media/docs/cute/...) if you want to learn more about high performance gemm. OpenAI triton is another good resource if you want to write relatively performant cuda kernels using python for deep learning (https://openai.com/research/triton https://openai.com/research/triton)

gpuhacker 3y ago

If you want to go really in-depth I can recommend GTC on demand. It's Nvidia streaming platform with videos from past GTC conferences. Tony Scuderio had a couple of videos on there called GPU memory bootcamp that are among the best advanced GPU programming learning material out there.

zetazzed 3y ago
100% this. You can find all kinds of detailed topics, like CUDA graphs, memory layout optimization, optimizing storage access, etc. https://www.nvidia.com/en-us/on-demand/ https://www.nvidia.com/en-us/on-demand/. They have "playlists" for things like HPC or development tools that collect the most popular videos on those topics.