5 ms·

GPT-3 doesn't care about anything except predicting the next token. It learned something about structure and meaning in the process.

pwinnski 6y ago
About structure, clearly. About meaning, not so clear. It seems more supportable to say that GPT-3 knows nothing about meaning, but that its knowledge of structure often gives an illusion of meaning.
- armitron 6y ago
  How is that different from how we use language besides our knowledge of structure being more layered and the abstractions more tight? What is meaning if not illusion?
- chillacy 6y ago
  This is the thesis of Bender 2020 https://www.aclweb.org/anthology/2020.acl-main.463.pdf https://www.aclweb.org/anthology/2020.acl-main.463.pdf when evaluating GPT-2. They distinguish between meaning and form. We don't _really_ know what the physical manifestations of meaning and form are in the brain... they're just concepts we invented. If anything, GPT-3 is suggesting that either: 1. Tasks which were previously thought to require meaning actually turn out only to require form. 2. Meaning and form are more related than previously thought. Both are interesting findings imo, but 2 would be huge, especially if it suggests how the brain might work. Could meaning be an emergent phenomenon of form?
  - canjobear 6y ago
    It seems like Bender & Koller's argument can just as easily prove that humans don't have representations of meaning either. Our brains get input from multiple "languages"---conventional natural languages as well as the neural codes that represent our sensory inputs. All we have access to is the form of these inputs, and so we can never learn the underlying "meaning".
    - visarga 6y ago
      Meaning comes from the game, not from the brain. It's the relation between agent and environment that creates meaning.
      - chillacy 6y ago
        The paper makes reference to the Symbol Grounding Problem, but I have not found the SGP's distinction between form and meaning to be completely convincing without some evidence of a physical, observable process. At the end when you look long enough it seems to call into question the very nature of consciousness.
    - chillacy 6y ago
      Hmm I think I see what you're saying. Reality is experienced through electrical signals, arguably we're just learning those inputs. But then I (as always I can only speak for myself, everyone else could be a p-zombie for all I know!) also have a qualitative experience of trees, and words, and an experience of meaning and understanding. If you look too deeply it quickly gets philosophical. https://en.wikipedia.org/wiki/Hard_problem_of_consciousness https://en.wikipedia.org/wiki/Hard_problem_of_consciousness
      - vladTheInhaler 6y ago
        > Reality is experienced through electrical signals, arguably we're just learning those inputs. That's exactly what we're doing. And we're never given the "answer sheet" to figure out whether we understood the platonic, capital T Truth, or whether we just learned a spurious correlation. We just keep getting more of those inputs. Which is why it seems to me that an unsupervised sequence prediction model like GPT-3 is the only sort that could ever give rise to something akin to human consciousness. The big differentiator seems to be that with a pure text sequence model, inputs go in, but the outputs don't have any control over future inputs. It isn't structured to have anything like agency, just passive observation and prediction. But a useful "understanding" in a human sense is related to what can be done with that understanding to enact change in the environment. I don't know how you would teach it that without giving it a Reddit account and setting it loose. > But then I (as always I can only speak for myself, everyone else could be a p-zombie for all I know!) also have a qualitative experience of trees, and words, and an experience of meaning and understanding. If you look too deeply it quickly gets philosophical. I'm not so sure I have those things. I'm glad you do. That's one reason I'm never going to do ketamine.
- ma2rten 6y ago
  I'm not sure what your definition of meaning is. Take this example from the GPT3 paper: To do a "farduddle" means to jump up and down really fast. An example of a sentence that uses the word farduddle is: One day when I was playing tag with my little sister, she got really excited and she started doing these crazy farduddles. According to my understanding of the concept it must know something about meaning and is able to reason about it if it was able to generate this.
  - notahacker 6y ago
    Depends what you're talking about by meaning. It correctly interprets the first part of the prompt as 'farduddle ~= jump' and the second part as an instruction to generate a sentence containing farduddle, possibly utilising a corpus of existing sentences containing jump in the context of 'really fast'. But that's also a series of instructions you could imagine as a DSL a relatively simple program could parse and generate a satisfactory response to. Which I believe the OP is classing as 'structure' since it's just performing translations based on familiar syntax. Understanding the concept of 'jumping' is a step further, before we get into the more philosophical stuff about qualia and whether things that can't jump can ever truly understand the experience of jumping...
    - ma2rten 6y ago
      Right, in this context I define meaning as the relationship between symbols. Form would be how to structure a sentence (but that sentence wouldn't have to make sense). If the parent comment was making some kind of Chinese Room argument than I think that's not very helpful for the discussion. "GPT-3 learned nothing about meaning because no computer program can do that by definition". But I don't think they were trying to make that kind of argument as you seem to be suggesting because they said they were unsure if it learned anything about meaning.
  - YeGoblynQueenne 6y ago
    - ...And Morris? - Um, it actually already is a word. Tnetennba. - Good heavens, really? Could you, uh, use it in a sentence for us? - "Good morning. That's a nice Tnetennba" [1]. ______________ [1] Moss from IT Crowd on Countdown: https://youtu.be/g9ixvD0_CmM?t=52 https://youtu.be/g9ixvD0_CmM?t=52 Edit: to clarify, if you don't know what a word means, just seeing it used in a sentence won't necessarily tell you much about its meaning, so that a language model was able to generate a phrase with the word in it doesn't necessarily tells us it understands the word's meaning.
- visarga 6y ago
  The old debate about syntax and semantics. I would say syntax, when embodied in the world gains meaning. Meaning comes from the context of the agent and its goals.

pwinnski 6y ago

About structure, clearly. About meaning, not so clear. It seems more supportable to say that GPT-3 knows nothing about meaning, but that its knowledge of structure often gives an illusion of meaning.

armitron 6y ago
How is that different from how we use language besides our knowledge of structure being more layered and the abstractions more tight? What is meaning if not illusion?
chillacy 6y ago
This is the thesis of Bender 2020 https://www.aclweb.org/anthology/2020.acl-main.463.pdf https://www.aclweb.org/anthology/2020.acl-main.463.pdf when evaluating GPT-2. They distinguish between meaning and form. We don't _really_ know what the physical manifestations of meaning and form are in the brain... they're just concepts we invented. If anything, GPT-3 is suggesting that either: 1. Tasks which were previously thought to require meaning actually turn out only to require form. 2. Meaning and form are more related than previously thought. Both are interesting findings imo, but 2 would be huge, especially if it suggests how the brain might work. Could meaning be an emergent phenomenon of form?
- canjobear 6y ago
  It seems like Bender & Koller's argument can just as easily prove that humans don't have representations of meaning either. Our brains get input from multiple "languages"---conventional natural languages as well as the neural codes that represent our sensory inputs. All we have access to is the form of these inputs, and so we can never learn the underlying "meaning".
  - visarga 6y ago
    Meaning comes from the game, not from the brain. It's the relation between agent and environment that creates meaning.
    - chillacy 6y ago
      The paper makes reference to the Symbol Grounding Problem, but I have not found the SGP's distinction between form and meaning to be completely convincing without some evidence of a physical, observable process. At the end when you look long enough it seems to call into question the very nature of consciousness.
  - chillacy 6y ago
    Hmm I think I see what you're saying. Reality is experienced through electrical signals, arguably we're just learning those inputs. But then I (as always I can only speak for myself, everyone else could be a p-zombie for all I know!) also have a qualitative experience of trees, and words, and an experience of meaning and understanding. If you look too deeply it quickly gets philosophical. https://en.wikipedia.org/wiki/Hard_problem_of_consciousness https://en.wikipedia.org/wiki/Hard_problem_of_consciousness
    - vladTheInhaler 6y ago
      > Reality is experienced through electrical signals, arguably we're just learning those inputs. That's exactly what we're doing. And we're never given the "answer sheet" to figure out whether we understood the platonic, capital T Truth, or whether we just learned a spurious correlation. We just keep getting more of those inputs. Which is why it seems to me that an unsupervised sequence prediction model like GPT-3 is the only sort that could ever give rise to something akin to human consciousness. The big differentiator seems to be that with a pure text sequence model, inputs go in, but the outputs don't have any control over future inputs. It isn't structured to have anything like agency, just passive observation and prediction. But a useful "understanding" in a human sense is related to what can be done with that understanding to enact change in the environment. I don't know how you would teach it that without giving it a Reddit account and setting it loose. > But then I (as always I can only speak for myself, everyone else could be a p-zombie for all I know!) also have a qualitative experience of trees, and words, and an experience of meaning and understanding. If you look too deeply it quickly gets philosophical. I'm not so sure I have those things. I'm glad you do. That's one reason I'm never going to do ketamine.
ma2rten 6y ago
I'm not sure what your definition of meaning is. Take this example from the GPT3 paper: To do a "farduddle" means to jump up and down really fast. An example of a sentence that uses the word farduddle is: One day when I was playing tag with my little sister, she got really excited and she started doing these crazy farduddles. According to my understanding of the concept it must know something about meaning and is able to reason about it if it was able to generate this.
- notahacker 6y ago
  Depends what you're talking about by meaning. It correctly interprets the first part of the prompt as 'farduddle ~= jump' and the second part as an instruction to generate a sentence containing farduddle, possibly utilising a corpus of existing sentences containing jump in the context of 'really fast'. But that's also a series of instructions you could imagine as a DSL a relatively simple program could parse and generate a satisfactory response to. Which I believe the OP is classing as 'structure' since it's just performing translations based on familiar syntax. Understanding the concept of 'jumping' is a step further, before we get into the more philosophical stuff about qualia and whether things that can't jump can ever truly understand the experience of jumping...
  - ma2rten 6y ago
    Right, in this context I define meaning as the relationship between symbols. Form would be how to structure a sentence (but that sentence wouldn't have to make sense). If the parent comment was making some kind of Chinese Room argument than I think that's not very helpful for the discussion. "GPT-3 learned nothing about meaning because no computer program can do that by definition". But I don't think they were trying to make that kind of argument as you seem to be suggesting because they said they were unsure if it learned anything about meaning.
- YeGoblynQueenne 6y ago
  - ...And Morris? - Um, it actually already is a word. Tnetennba. - Good heavens, really? Could you, uh, use it in a sentence for us? - "Good morning. That's a nice Tnetennba" [1]. ______________ [1] Moss from IT Crowd on Countdown: https://youtu.be/g9ixvD0_CmM?t=52 https://youtu.be/g9ixvD0_CmM?t=52 Edit: to clarify, if you don't know what a word means, just seeing it used in a sentence won't necessarily tell you much about its meaning, so that a language model was able to generate a phrase with the word in it doesn't necessarily tells us it understands the word's meaning.
visarga 6y ago
The old debate about syntax and semantics. I would say syntax, when embodied in the world gains meaning. Meaning comes from the context of the agent and its goals.