Others have given good answers, I'd just add that from a lay-perspective you can consider it a kind of compression: If you're also familiar with generative ima

Others have given good answers, I'd just add that from a lay-perspective you can consider it a kind of compression: If you're also familiar with generative image models, you know there are models where you can type a sentence like "an astronaut on a horse" and it will draw you a pic. You can see how you could (imperfectly) compress a bunch of images by writing a caption for them, and uncompress them by running the caption through the generative model. The analogy holds for language models. You can compress a story about the assassination of Abraham Lincoln into the phrase "write me an account of the assassination of Abe Lincoln" and it will "uncompress" it with a common knowledge account of that event. Behind the scenes, the actual mechanism is different as others have mentioned, but the effect can be framed as I describe. Its basically filling in the gaps with common sense as learned from the text it was trained on.