Not really. Wittgenstein wasn’t making the radical claim that words just float around in utterances, with no connection to ideas in a human mind. He saying tha

Not really. Wittgenstein wasn’t making the radical claim that words just float around in utterances, with no connection to ideas in a human mind. He saying that language doesn’t reduce down to simple “atomic” forms, as he had previously thought. Much of speech is embedded in context in the way we live. You couldn’t take “I CAN HAZ CHEEZBURGER?” and reduce it down to something just by looking at the words and their relation to each other. The person using this statement is certainly not saying “For all elements of the set cheeseburger, there exists in the set of all possible futures one element which is proper to the set of elements identical to myself”. They are invoking a context, a mode of thinking, even a shared memory. This is why the meme is hilarious in some contexts, but would be utterly unintelligible in others. The existence of LLMs does not provide support for Wittgenstein’s theories, which are more about observing humans - it’s sometimes described as an “anthropological” theory of language. The fact that it’s an impersonal process doesn’t add much. Humans can find random poetry assembled with fridge magnets meaningful too, but that says little about the nature of language. PS Also, it’s well known that embeddings do seem to have a kind of “conceptual” relationship to each other. Famously some people find that the vector difference of king->queen will usually be related to man->woman. So, in some bizarre high dimensional way they do sort of have concepts. And given enough data they also have contexts. It’s extremely unclear if this is analogous to how humans actually think, but maybe LLMs have some facilities that are similarly powerful.