>This seems a bit tautological to me I meant it in the sense of: given any reasonable definition of "understanding", finding a mapping between the sequences of

>This seems a bit tautological to me I meant it in the sense of: given any reasonable definition of "understanding", finding a mapping between the sequences of words and the structure of the world must satisfy the definition. >I think it is a fairly common intuition that one cannot understand something just by rote-learning a bunch of facts. I agree, but its important to understand why this is. The issue is that learning an assignment between some words and some objects misses the underlying structure that is critical to understanding. For example, one can point out the names of birds but know nothing about them. It is once you can also point out details about their anatomy, how they interact with their environment, find food, etc and then do some basic reasoning using these bird facts that we might say you understand a lot about birds. The assumption underlying the power of these language models is that a large enough text corpus will contain all these bird facts, perhaps indirectly through being deployed in conversation. If it can learn all these details, deploy them correctly within context, and even do rudimentary reasoning using such facts (there are examples of GPT-3 doing this), then it is reasonable to say that the language model captures understanding to some degree.