This MIT lecture on recurrent neural networks provides a lot of context on why attention was so groundbreaking, and how things were solved before: https://www.y

This MIT lecture on recurrent neural networks provides a lot of context on why attention was so groundbreaking, and how things were solved before: https://www.youtube.com/watch?v=ySEx_Bqxvvo https://www.youtube.com/watch?v=ySEx_Bqxvvo And for the Attention Is All You Need paper itself, Andrej Karpathy's "GPT From Scratch" talk is a really great walkthrough of how to make this work in practice, all condensed into two hours of live coding: https://www.youtube.com/watch?v=kCc8FmEb1nY https://www.youtube.com/watch?v=kCc8FmEb1nY