"They predict the next word in a sequence based on correlations derived from a lot of training data." This misinformation really needs to finally die. They ar

"They predict the next word in a sequence based on correlations derived from a lot of training data." This misinformation really needs to finally die. They are trained based on correlations of next token prediction. But that does not mean that the resulting neural network is only doing that. When Harvard and MIT researchers fed Othello moves into GPT, it was training based on the correlations of legal Othello moves. But the version of the NN that performed the best at that task had created within that network dedicated structure representing a legal Othello board and tracking the state of moves - neither of which were things explicitly in the training data or goals the training was directly measuring. You effectively came to exist by a training process of surviving to reproduce. But it wouldn't exactly be accurate to say that as complexity increased, the only thing you remain capable of is surviving to reproduce, even if many of your emergent capabilities happen to improve the success of being able to do so. Training =/= Operation