I see the point you're trying to make, but you're being a bit too hyperbolic. The entire decoder is a loop over a function that takes context and generates some

I see the point you're trying to make, but you're being a bit too hyperbolic. The entire decoder is a loop over a function that takes context and generates some next-word probabilities. It's a very complicated function, certainly, but it literally is solving things by "get[ting] a bunch of text and calculat[ing] the most often used next word". What's impressive is how much it can do by just doing that, because that function is so complicated. But it clearly has limits that aren't related to the scope of the training data, as is demonstrated to me daily by getting into a circular argument with ChatGPT about something.