2 ms·
People do try to teach computers the basics - they just don't get very far. My optimism is kind of based on what people have been successful with eg. spatial st
by tim333 6y ago
People do try to teach computers the basics - they just don't get very far. My optimism is kind of based on what people have been successful with eg. spatial stuff with self driving cars, language with GTP3, face recognition, games like go etc. They seem to be gradually working though human abilities. Obvs there's stuff they are not so good at yet also.
- tsimionescu 6y agoI was specifically thinking of language here: GPT-3 does not "understand language" in any real sense. It is simply able to string together words that are more likely to come together in certain orders, but they are utterly meaningless to the network. Language is a mechanism that we use to express a thought (sometimes externally, but most of the time internally in our own minds). A major part of language is an understanding of the world you are "talking" about. And yet, GPT-3 has no idea about the world, it just knows that the words "the sky is blue" are more likely to appear in a corpus than "the sky is yellow". Even more deeply, it has no notion of what an object is, and how it might differ from an agent, what movement is etc - any of the things that make up our shared human model of the world, that we talk about. Until we can codify such a model (or automatically learn it), we won't be able to say that we have a model that "understands language". The GPT people are not even trying to look in this direction: they are throwing more text and more neurons at the problem and claiming it will work. They'll probably end up with a model that can essentially do the equivalent of instantly finding the best Shakespeare quote for any situation you throw at it, but still won't be able to reason about that situation in any way, or keep a story straight. This is already visible in GPT-3: its output (even though most of what you tend to see is carefully hand-picked out of a soup of more meaningless output) meanders around without any point to make: it simply does not have a point to make, and it cannot have a point, by the very nature of its construction. Essentially, language is words + context, and GPT-X only has access to the words, with none of the context. And furthermore, no one is even working on thinking to add the context yet.