4 ms·

Does GPT-4 Understand What We’re Saying?

4 points by vidyesh 4y ago

retrac 4y ago
> There’s a kind of understanding which is just coordination. For example, I could say, “Can you pass me the spoon?” And you’d say, “Here it is.” And I say, “Well, you understood me!” Because we coordinated. That’s not the same as generative or constructive understanding, where I say to you, “I’m going to teach you some calculus, and you get to use that knowledge on a problem that I haven’t yet told you about.” I think hang-up -- the automatic insistence on understanding and projection of anthropomorphic traits -- is that our instincts strongly tell us that mere statistical coordination shouldn't be capable of such feats. Maybe the closest analogy is how simple game space search could produce software much better than us playing chess. Something that seemed like it required human thought could be fully mechanized, and mechanized in a way that humans do not think, and yet achieve superior results. We're used to it now, but it must have unsettled people ~50 years ago in a similar way to how ChatGPT does. I consider it a very pressing problem in philosophy, actually. There's the concept of the "philosophical zombie", which is a being which would look like, talk like, and by all external indicators, sound, think, and feel like a human being. But which has no sentience. The lights aren't on. It's a pure automaton. Some philosophers are sort of torn on whether the notion even makes sense, and is properly framed. But among those who do think it is a properly defined idea, it's roughly split, according to a survey I saw a few years ago [1], on whether they are, in theory, metaphysically possible. Is there something about consciousness that is fundamental to explaining everything humans can do? Would the laws of nature, philosophy, the mind, whatever, allow such a thing to exist? I think LLMs tend to hint somewhat at their metaphysical possibility, which is rather disturbing to many people. It's much more comforting to project understanding on to it, than to envision the possibility of mindless human-like intelligence. [1] https://philpapers.org/archive/BOUWDP https://philpapers.org/archive/BOUWDP
- smoldesu 4y ago
  You're definitely onto something here. I look at it from a realist take - it's just language. Language can do impressive things and have impressive consequences, but they aren't sticks and stones. We ascribe high value to well-written text because it's seen as this token of intelligence, but on the other side of the board it's a game of pattern-recognition. We may well impress ourselves by how much work can be saved with guessing the rest of your response. As you say, the original sin is anthropomorphizing AI. We all want to do it, because it's a convenient way of looking at it and extremely gratifying to our inner science-fiction nerd. It has the unfortunate side-effect of implying something has a soul, or a lived experience that is the sum of something greater than text. It's not. Even AI trained on text and images and video is not capable of simulating a human life. The only way to accurately model the success, failure, longing and achievement of humankind is by growing into it one year at a time. Claims of "superhuman" writing speeds may well be true, and the text might even be written inscrutably well. It's not human, though, and humanity is a poor benchmark for LLM scientists that ought to know better.

4 points by vidyesh 4y ago

> There’s a kind of understanding which is just coordination. For example, I could say, “Can you pass me the spoon?” And you’d say, “Here it is.” And I say, “Well, you understood me!” Because we coordinated. That’s not the same as generative or constructive understanding, where I say to you, “I’m going to teach you some calculus, and you get to use that knowledge on a problem that I haven’t yet told you about.” I think hang-up -- the automatic insistence on understanding and projection of anthropomorphic traits -- is that our instincts strongly tell us that mere statistical coordination shouldn't be capable of such feats. Maybe the closest analogy is how simple game space search could produce software much better than us playing chess. Something that seemed like it required human thought could be fully mechanized, and mechanized in a way that humans do not think, and yet achieve superior results. We're used to it now, but it must have unsettled people ~50 years ago in a similar way to how ChatGPT does. I consider it a very pressing problem in philosophy, actually. There's the concept of the "philosophical zombie", which is a being which would look like, talk like, and by all external indicators, sound, think, and feel like a human being. But which has no sentience. The lights aren't on. It's a pure automaton. Some philosophers are sort of torn on whether the notion even makes sense, and is properly framed. But among those who do think it is a properly defined idea, it's roughly split, according to a survey I saw a few years ago [1], on whether they are, in theory, metaphysically possible. Is there something about consciousness that is fundamental to explaining everything humans can do? Would the laws of nature, philosophy, the mind, whatever, allow such a thing to exist? I think LLMs tend to hint somewhat at their metaphysical possibility, which is rather disturbing to many people. It's much more comforting to project understanding on to it, than to envision the possibility of mindless human-like intelligence. [1] https://philpapers.org/archive/BOUWDP https://philpapers.org/archive/BOUWDP

smoldesu 4y ago
You're definitely onto something here. I look at it from a realist take - it's just language. Language can do impressive things and have impressive consequences, but they aren't sticks and stones. We ascribe high value to well-written text because it's seen as this token of intelligence, but on the other side of the board it's a game of pattern-recognition. We may well impress ourselves by how much work can be saved with guessing the rest of your response. As you say, the original sin is anthropomorphizing AI. We all want to do it, because it's a convenient way of looking at it and extremely gratifying to our inner science-fiction nerd. It has the unfortunate side-effect of implying something has a soul, or a lived experience that is the sum of something greater than text. It's not. Even AI trained on text and images and video is not capable of simulating a human life. The only way to accurately model the success, failure, longing and achievement of humankind is by growing into it one year at a time. Claims of "superhuman" writing speeds may well be true, and the text might even be written inscrutably well. It's not human, though, and humanity is a poor benchmark for LLM scientists that ought to know better.