Personally I find value in toy examples like this as it helps me check I actually understand the core architecture. You can look at a diagram or an equation exp

Personally I find value in toy examples like this as it helps me check I actually understand the core architecture. You can look at a diagram or an equation explaining a transformer block but how do you make the jump from that to actually implementing it? With such examples you can check your understanding of what's actually getting computed where, what the dimensions of the vectors and matrices are etc. Sure the vast majority of people who want to work with LLMs don't need to know any of this. But personally I enjoy understanding how things work and I like to be able to drill down into what's happening at a very basic level. Something like this is a lot easier to play around with and check your understanding vs the source code of a real transformer model. Once you feel you've mastered a toy example like this, the real code becomes easier to get to grips with.