Tangential to this post but Peter Bloem’s Transformers from Scratch (2019) was one of the best early tutorials on transformers and self-attention. There have be

Tangential to this post but Peter Bloem’s Transformers from Scratch (2019) was one of the best early tutorials on transformers and self-attention. There have been many more tutorials since then but this was the one that solidified a mental picture for me. https://peterbloem.nl/blog/transformers https://peterbloem.nl/blog/transformers