There's been so much drift in what people expect from a language model. We used to expect a language model would tell you which sentences were likely and which

There's been so much drift in what people expect from a language model. We used to expect a language model would tell you which sentences were likely and which were unlikely, which things were grammatical and which were not -- but this wasn't initially expected to be tied to a detailed knowledge of the world or general reasoning ability. With GPT3, we've seen people prompt it to generate tables of factual information (e.g. state populations), and commenters can simultaneously be surprised that some of the facts are on the right scale, and also disappointed that they're wrong. Here, an AI researcher has to argue that a model trained only on text hasn't learned about physics or geometry or social norms or a bunch of other stuff that we wouldn't assume is well captured in just whatever text is available. I think maybe the fault is not that GPT3 doesn't know these things. The fault is that as humans, we're so dependent on language both for communication with others and also for our own cognition, that when we encounter a really good language model, it's hard for us to _not_ see some glimmer of general AI. We're so impressed that we unreasonably move the goal posts. https://www.smbc-comics.com/comic/ball https://www.smbc-comics.com/comic/ball And it's worth asking -- we consider a human speaker to "know" a language when they've internalized its grammar and vocabulary, but not all that specific stuff about the world. An 18th century English speaker and a 21st century English speaker were/are aware of drastically different facts, and are likely to produce different sentences, but there's something about English that they both know. Not as a criticism of GPT3 but as a question about NLP researchers -- why can we not isolate and represent that in a model?