5 ms·
I made these a couple of years ago as a teaching exercise for https://minitorch.github.io/ https://minitorch.github.io/. At the time the resources for doing any
by srush 2y ago
I made these a couple of years ago as a teaching exercise for https://minitorch.github.io/ https://minitorch.github.io/. At the time the resources for doing anything on GPUs were pretty sparse and the NVidia docs were quite challenging.
These days there are great resources for going deep on this topic. The CUDA-mode org is particularly great, both their video series and PMPP reading groups.
- olive247 2y agoDo you have links to the other great resources you are referring to?
- bytepoet 2y agoThanks a lot, Sasha, for creating these. I found your LLM training puzzles to be excellent as well.
- srush 2y agoAwesome! Here are all of them if anyone else is looking. https://github.com/srush/Triton-puzzles https://github.com/srush/Triton-puzzles https://github.com/srush/tensor-puzzles https://github.com/srush/tensor-puzzles https://github.com/srush/autodiff-puzzles https://github.com/srush/autodiff-puzzles https://github.com/srush/transformer-puzzles https://github.com/srush/transformer-puzzles https://github.com/srush/GPTworld https://github.com/srush/GPTworld https://github.com/srush/LLM-Training-Puzzles https://github.com/srush/LLM-Training-Puzzles
- nextos 2y agoSlightly offtopic, but any chance you could update or re-upload code for your https://github.com/harvardnlp/DeepLatentNLP https://github.com/harvardnlp/DeepLatentNLP tutorial? I found the NLP latent variable models discussed there really interesting, and notebooks were excellent. However, these seem gone and the only thing left are slides? Alternatively, any other places that discuss the same topics, including some code? I could only find equivalent discussions with code in Pyro docs and Kevin Murphy's book, volume 2. But these are more sparse as they also cover many other topics.
- lins1909 2y agoThanks Sasha - this looks like a great resource.Just to be clear, would you recommend going through other newer resources than this instead? Not sure if your comment is to discourage someone from going through this.