I don't think there is a magic alrogithm at play. I think it's all about studying the data - the geometry of the data manifold. In the case of a transformer mor

I don't think there is a magic alrogithm at play. I think it's all about studying the data - the geometry of the data manifold. In the case of a transformer more precisely learning the geometry of the stochastic process of tokens. Common "patterns", such as repetition or symmetry, appear there and can be learned. I think like learning eigenvectors in PCA you just learn more and more patterns and combinations of patterns. Then at some point that starts looking like pattern recognition.