Yann LeCun gave a recent talk, at 10:30 he gives a good explanation that doesn't appeal to "what LLMs are trying to do on the inside": LeCun just observes that

Yann LeCun gave a recent talk, at 10:30 he gives a good explanation that doesn't appeal to "what LLMs are trying to do on the inside": LeCun just observes that next-token prediction is exponentially diverging, since every next token with the smallest error e is still (roughly speaking) (1-e)^n for output of length n. I just thought it was a good complexity-theory based explanation (unlike most other statistical-parrot type arguments). It is a single slide, very helpful: https://www.youtube.com/watch?v=ETZfkkv6V7Y https://www.youtube.com/watch?v=ETZfkkv6V7Y