> There's no doubt that an LLM can uncover whatever is structurally included in its training data, even if that encoding is implicit. What's less believable is

> There's no doubt that an LLM can uncover whatever is structurally included in its training data, even if that encoding is implicit. What's less believable is that the LLM somehow achieves a grounded understanding of physical aspects of experience purely from streams of raw text. I was responding to the point in your first paragraph in your upper level comment and this quoted paragraph. Multimodal models will address that. Your new point about perception could still be a problem for multimodal models, but it’s also possible that with sufficient data, the new models could address that (eg it could say people perceive the colors differently, as A and B). Multimodal models map between human language and raw sensory data. It’s fundamentally similar, in principle, to part of how we learn our first language. (Not saying we do only that though.)