2 ms·
I really like Andrej Karpathys content yeah its youtube i know... but, its hand on too for gpt/llm/ml in general https://karpathy.ai/zero-to-hero.html https:/
by tayo42 3y ago
I really like Andrej Karpathys content
yeah its youtube i know... but, its hand on too
for gpt/llm/ml in general https://karpathy.ai/zero-to-hero.html https://karpathy.ai/zero-to-hero.html
it starts with writing back prop from scratch and take you through writing everything you need and training a gpt2 equivalent model in the end
I also thought his lectures at standford, on youtube cs231n 2016, were really good. they cover GAN for generation, but I think after that you can read the papers that were the source for diffusion models dalle and midjouney use
- rmshin 3y agoI just wanted to second this, I recently went through all of Karpathy's videos from almost zero baseline ML knowldege and am now fairly comfortable writing language models from scratch (simple bigram statistical models, MLPs, transformers, etc.). Karpathy is quite good at helping you build an intuitive understanding of core concepts and linking/referencing literature where appropriate for the more curious learners. Thanks to his videos, I was able to read through several of the foundational papers on resnet, convnet, transformers, and some misc. normalisation techniques without _too_ much struggle. At one point in time I also went through half of Andrew Ng's CS229 ML lectures from Stanford (https://www.youtube.com/watch?v=jGwO_UgTS7I&list=PLoROMvodv4rMiGQp3WXShtMGgzqpfVfbU https://www.youtube.com/watch?v=jGwO_UgTS7I&list=PLoROMvodv4...). Found them much more math/proof heavy, but definitely valuable for understanding the underlying statistical methods & theory that ML apply.