This is a great comprehensive deep dive, thank you for sharing! Interestingly, there was a blog post and video in 2019 by Peter Bloem [1] with the same title w

This is a great comprehensive deep dive, thank you for sharing! Interestingly, there was a blog post and video in 2019 by Peter Bloem [1] with the same title which I consider to be one of the very best quick intros to transformers. Rather than diving into all of the details right away, Peter focuses on the intuition and “peels the onion” gradually. [1] http://peterbloem.nl/blog/transformers http://peterbloem.nl/blog/transformers Other notable good intros to transformers are: The Illustrated Transformer by Jay Alammar https://jalammar.github.io/illustrated-transformer/ https://jalammar.github.io/illustrated-transformer/ The Annotated Transformer by Harvard NLP https://nlp.seas.harvard.edu/2018/04/03/attention.html https://nlp.seas.harvard.edu/2018/04/03/attention.html