That's something I've been suspecting for a while. More powerful models produce overfitting, so it is likely that GPT-2 is simply memorizing whole texts and the

That's something I've been suspecting for a while. More powerful models produce overfitting, so it is likely that GPT-2 is simply memorizing whole texts and then regurgitating them in whole. Knowing how they're generated, a sequence of sentences that make sense are likely copied almost verbatim from an article written by a human. Without understanding the concepts, the algorithm may simply repeat words that go well together - and what goes together better than sentences that were written together in the first place? What the GPT model is really good at is at is identifying when a sentence makes sense in the current context. Given that it has half of the internet as its learning corpus, it is easy that it's simply returning a piece of text that we do not know about. The real achievement thus is finding ideas that are actually appropriate in relation to their input text.