> my question is if you, the human, can learn to understand what it is to perceive magnetic fields? I think obviously the answer is yes. I certainly don't see

> my question is if you, the human, can learn to understand what it is to perceive magnetic fields? I think obviously the answer is yes. I certainly don't see that as obvious, and I would guess that while you can learn _about_ their perceptual mode, you can't learn what it is like to perceive magnetic fields just through talking about it. I would consider the Mary's Room thought experiment, and the What Is It Like To Be a Bat paper from Nagel. I think there's a relationship to the Chinese Room, but I want to be clear. In the original formulation, the person in the room follows a book of pre-provided instructions to produce a response. The LLM and person in the Thai text completion scenario must learn an equivalent set of instructions themselves, and for this I would claim that they are comparable to the human + book combination in the original Chinese Room. The person who learns to complete Thai text doesn't know what they're talking about, but they know more than the person following instructions in the Chinese Room. But clearly they still don't know what a Thai speaker knows. > I guess it just feels like this general argument, that merely seeing things and making predictions that turn out to be right isn't enough to understand it will never go away. We could have a full fledged robot walking around having conversations and I could dispute its ability to really understand. No, perhaps the end of my original statement didn't make this clear, but I think AI systems _can_ know things, and knowing is not a binary but part of a range. StabilityAI / DALL-e know quite a bit about the relationship between texts and images, and the structure within images -- but they _don't_ know about bodies, physical reality, etc etc. A system that has multiple modalities of perception, learns to physically navigate the world, interact with objects, make and execute plans by understanding the likely effects of actions, etc -- knows and understands a lot. I'm not arguing about a hard limitation of AI; I'm arguing about a limitation of the way our current AIs are built and trained.