3 ms·
"I’m teaching myself LLM internals by re-implementing the stack from first principles." - curious what resources you're using? Any books or courses, or just bui
by silentsea90 1y ago
"I’m teaching myself LLM internals by re-implementing the stack from first principles." - curious what resources you're using? Any books or courses, or just building it straight up? Great work!
- deleted 1y ago[deleted]
- matthewolfe 1y agoModal's GPU glossary is a good overview about how GPUs work [0]. Karpathy's LLM overview is a good high level overview on LLMs [1]. 3b1b's video (and subsequent videos) on transformers was excellent at helping me understand the math at a high level [2]. This matrix multiplication optimization worklog helped me understand writing better CUDA (not for beginner intro though) [3]. During this process I also asked ChatGPT a lot of questions. I'm definitely open to suggestions about "how to learn" with all the new tools we have. I felt this has not been straightforward to figure out. [0] https://modal.com/gpu-glossary https://modal.com/gpu-glossary [1] https://www.youtube.com/watch?v=7xTGNNLPyMI https://www.youtube.com/watch?v=7xTGNNLPyMI [2] https://www.youtube.com/watch?v=wjZofJX0v4M https://www.youtube.com/watch?v=wjZofJX0v4M [3] https://siboehm.com/articles/22/CUDA-MMM https://siboehm.com/articles/22/CUDA-MMM