I don't think that's where the difference lies, no. Sampling from a markov model built from English n-gram frequencies will produce non-repeated, mostly grammat

I don't think that's where the difference lies, no. Sampling from a markov model built from English n-gram frequencies will produce non-repeated, mostly grammatical English text, which frequently is even sensible-sounding once n >= 4 or so. But I don't think there's anything going on in language generation beyond "based on the current context, produce an appropriate output token for that context based on the observed and inferred distribution of training inputs". I think that's also how human language generation works, though "the context" for humans includes a lot more than just a few thousand words of text. But I think that the surprising thing about e.g. GPT-4 is how well it does the thing, rather than the fact that it does the thing at all.