3 ms·

I recently decided to turbo-teach myself basic cuda with a proper project. I really enjoyed Karpathy’s micrograd (https://github.com/karpathy/micrograd https://

I recently decided to turbo-teach myself basic cuda with a proper project. I really enjoyed Karpathy’s micrograd (https://github.com/karpathy/micrograd https://github.com/karpathy/micrograd), so I extended it with cuda kernels and 2D tensor logic. It’s a bit longer than the original project, but it’s still very readable for anyone wanting to quickly learn about gpu acceleration in practice.

quibono 3y ago
Is your work publicly available anywhere? I'd love to have a look if possible.
- cataPhil 3y ago
  Hi :) Yes it's in the post url or here: https://github.com/mlecauchois/micrograd-cuda https://github.com/mlecauchois/micrograd-cuda
  - quibono 3y ago
    Oh sorry, I didn't realise you were OP :) Thanks!
lagrange77 3y ago
Awesome, thank you for publishing it. I was planning the same, but didn't find the time yet.
coolThingsFirst 3y ago
Lovely project, what did you find as the beat place to learn about cuda?
- cataPhil 3y ago
  Thanks! I highly recommend the book Programming Massively Parallel Processors
  - st380752143 3y ago
    This is the second time this week that people have recommended this boot while I was searching for information about CUDA. Thanks!
lairv 3y ago
If you ever feel like improving your matmul kernel you can check this blog https://siboehm.com/articles/22/CUDA-MMM https://siboehm.com/articles/22/CUDA-MMM
- cataPhil 3y ago
  Thanks! I will probably do that this weekend