I mean this is the limit of current state of the art AI, right? We're seeing with things like GPT3 that an actual powerful language model needs to have a load

I mean this is the limit of current state of the art AI, right? We're seeing with things like GPT3 that an actual powerful language model needs to have a load of real world knowledge built in. But that knowledge is all based on experiences by others. From the way it talks (and generates poetry) you can tell that it is unable to synthesize new experiences. It can't come up with a description (or metaphor) of human experience that it hasn't been taught. To be a convincing AI, you don't just need to replicate descriptions of experience from memory, but actually have new experiences as well. Meaning that the Chinese room, in order to be able to demonstrate "understanding", will also require other inputs than just text. Otherwise it can't really understand the world, because it only knows about the world from hear-say.