Thank you for sharing! For the "from scratch" version, I recommend "The GPT-3 Architecture, on a Napkin" https://dugas.ch/artificial_curiosity/GPT_architecture

Thank you for sharing! For the "from scratch" version, I recommend "The GPT-3 Architecture, on a Napkin" https://dugas.ch/artificial_curiosity/GPT_architecture.html https://dugas.ch/artificial_curiosity/GPT_architecture.html, which was there as well (https://news.ycombinator.com/item?id=33942597 https://news.ycombinator.com/item?id=33942597). Then, to actually dive into details, "The Annotated Transformer", i.e. a walktrough "Attention Is All You Need", with code in PyTorch, https://nlp.seas.harvard.edu/2018/04/03/attention.html https://nlp.seas.harvard.edu/2018/04/03/attention.html.