> Nope, there's no reasoning. It's just generating the text that best matches its training data. That's like saying that when you answer questions on an exam,

> Nope, there's no reasoning. It's just generating the text that best matches its training data. That's like saying that when you answer questions on an exam, you're just generating the text that best matches your training data... Both statements are correct, but only if you understand what "generating" and "matches" mean. Generating doesn't (always) mean copying, and matches doesn't (always) mean exactly the same. In the more general case you're drawing a kind of analogy between what you were taught and the new problem you are answering. You should google "Induction heads" which is one of the mechanisms that researchers believe Transformers are using to perform in-context learning. In the general case this is an analogical A'B' => AB type of "prediction".