5 ms·

> What are humans except "text generation" machines? The ability to grow and choose our own direction: to choose what our goals are, curiosity, self-awareness,

> What are humans except "text generation" machines? The ability to grow and choose our own direction: to choose what our goals are, curiosity, self-awareness, desire. To imply that GPT-3 is anything close to strong AI is kind of ridiculous.

quotemstr 6y ago
And how do we indicate that any of these processes has occurred except through the medium of language? A sufficiently good text predictor is a sentient mind. I don't believe that there's any experience of consciousness distinct from the use of language.
- drcode 6y ago
  I disagree with this... I have a hunch GPT-3 is still falling short PRECISELY because of its dependence on language, and that it actually is going to great lengths to overcome this design flaw to create the simple texts that we're all fawning over. I predict within a few years, the descendants of GPT-3 will use very different fundamental units for processing that differ greatly from the current state-of-the-art (i.e. they won't use BPEs and their ilk anymore, except for final output) and will be far more powerful as a result.
  - Findeton 6y ago
    The descendants of GPT-3 will be using the comments on this page to get new ideas. I do agree with you. We, as somewhat intelligent beings, do not base our thinking on words or language AFAIK, even though it's our best way to convey ideas to others. And we learn through experience, way faster than GPT-3 does, with fewer shots. It looks like the attention mechanisms are what made these models actually start to understand things... But those attention mechanisms are still very raw and mainly designed to be easy to execute on current hardware, I wonder how fast will we refine that. Finally it looks like, once trained, these models don't learn when we use them. It definitely doesn't learn through experience and that's a major limitation on how intelligent it can be.
- patagurbon 6y ago
  I don't believe we have any evidence this edition of GPT is capable of reasoning. I haven't experimented with it but I doubt it will respond correctly to even simple logic puzzles provided they are framed in a novel way (it may have already seen some puzzles, but I doubt it can extend that knowledge to a puzzle with different words)
  - quotemstr 6y ago
    This version of GPT can add, subtract, multiply, and divide without ever having been taught to do these things things. Yes, it can reason.
- oehtXRwMkIs 6y ago
  For most of human history text has not existed. I think you are conflating language with written language which is quite common in the post-Gutenberg age. I think sentience like most things is a spectrum, so I'm not really sure what you mean by sentient, but I would argue that for most people the bar for sentience is much higher than text prediction. The Chinese room is only one aspect of our minds, and we don't even know what consciousness is.
  - quotemstr 6y ago
    I have to disagree with this perspective and reiterate my original position: language is sentience. And to be fair, reasonable people stake out positions on both sides of this debate: I'm not claiming that the alternative proposition is somehow unreasonable. It's a legitimate subject of scholarly disagreement. Nevertheless, I'm still firm on language. Why? Because all complexity is ultimately about symbolic manipulation of terms representing the process of manipulation itself. ("Godel, Escher, Bach" is a fantastic exploration of this concept.) How can you manipulate concepts without assigning terms to their parts? That's what language is. The question I like to ask is this: are there any ideas that you cannot express using language? No? Then how is thought distinct from language? Yes, people (myself included) experience a "tip of the tongue" experience where you feel like you have an idea you can't just yet express. But maybe this experience is what reason feels like. Why should idea formation take only one "clock cycle" in the brain? Why should we be unaware of the process? I think this feeling of having an idea yet being unable to formulate it is just the neural equivalent of a CPU pipeline stall. It's not evidence that we can have ideas without language: it's evidence that ideas sometime take a little while to gel.
    - oehtXRwMkIs 6y ago
      I think you're mixing up sentience/consciousness/intelligence. Many animals are sentient for example but as far as we know, they don't really have language. But I think I get what you're getting at, which I believe is "human level intelligence requires langauge". I think that's a reasonable take. But you said "sentience", and you said "is", which makes your position difficult to agree with.
- pastrami_panda 6y ago
  > I don't believe that there's any experience of consciousness distinct from the use of language. To me this indicates a very narrow view of consciousness. Consider for a moment the quiet consciousness of the cerebellum for example. I like the way David F. Wallace put it: 'Both flesh and not'. There's an astounding amount of consciousness that is not bound by language. One can even argue that language might hinder those forms of consciousness from even arising.
  - quotemstr 6y ago
    If you believe in consciousness divorced from flesh, you're in strong intellectual company, but that's not a path I can go down. Metaphysically, I just can't accept the idea that the mind is anything but the computation performed by the brain.
    - henearkr 6y ago
      I rather agree with you, but the interaction of the mind with the physical reality is also extremely important to shape it. GPT3 has no interaction with a physical world. Any formal system that cannot interact with something outside of itself will be intrinsically limited, for one thing by Gödel incompleteness theorem.
      - visarga 6y ago
        I agree about the need for an environment. The difference between GPT-3 and an agent in an environment is that GPT-3 only saw tons of static text, while an agent can design an action (an experiment), act it out and observe the results, drawing conclusions. Thus it can act in a way similar to the scientific method.
    - pastrami_panda 6y ago
      In the book the meaning of the quote is more to the effect of how truly great athletes can perform on the verge of what spectators would consider inhuman or possible. I'd be hard-pressed to believe that language and syntax would be responsible for these kinds of actions and flow. I'd argue that getting into such a state is not possible while the mind is caught up in the language of things rather than the experience itself, and reacting to it directly. This is what I meant by the quiet consciousness, devoid of language or syntax.
- jodrellblank 6y ago
  > I don't believe that there's any experience of consciousness distinct from the use of language. Not sure there's one I can communicate to you, but I'm perfectly capable of forgetting the word for something and still knowing unambiguously yet wordlessly what it is, that's an experience. Catching a ball? Running? Experiencing emotions from wordless music? Viewing scenery? Engaging with a computer game? How are they not conscious experiences?
  - jengleton 6y ago
    Also interesting to note that a good portion of people lack an internal monologue. This interview made the rounds a while back: https://www.youtube.com/watch?v=u69YSh-cFXY https://www.youtube.com/watch?v=u69YSh-cFXY
- visarga 6y ago
  > I don't believe that there's any experience of consciousness distinct from the use of language. What is the role of the body in consciousness, then?
visarga 6y ago
GPT-3 has flexible goals too. It can learn a new task and do it in the same step. What GPT3 doesn't have is a body.
- solidasparagus 6y ago
  No, you can teach it a new task or to do multiple tasks, but it will never be able to independently identify what a new task might be and learn to do it. An important distinction when talking about AGI.

quotemstr 6y ago

And how do we indicate that any of these processes has occurred except through the medium of language? A sufficiently good text predictor is a sentient mind. I don't believe that there's any experience of consciousness distinct from the use of language.

drcode 6y ago
I disagree with this... I have a hunch GPT-3 is still falling short PRECISELY because of its dependence on language, and that it actually is going to great lengths to overcome this design flaw to create the simple texts that we're all fawning over. I predict within a few years, the descendants of GPT-3 will use very different fundamental units for processing that differ greatly from the current state-of-the-art (i.e. they won't use BPEs and their ilk anymore, except for final output) and will be far more powerful as a result.
- Findeton 6y ago
  The descendants of GPT-3 will be using the comments on this page to get new ideas. I do agree with you. We, as somewhat intelligent beings, do not base our thinking on words or language AFAIK, even though it's our best way to convey ideas to others. And we learn through experience, way faster than GPT-3 does, with fewer shots. It looks like the attention mechanisms are what made these models actually start to understand things... But those attention mechanisms are still very raw and mainly designed to be easy to execute on current hardware, I wonder how fast will we refine that. Finally it looks like, once trained, these models don't learn when we use them. It definitely doesn't learn through experience and that's a major limitation on how intelligent it can be.
patagurbon 6y ago
I don't believe we have any evidence this edition of GPT is capable of reasoning. I haven't experimented with it but I doubt it will respond correctly to even simple logic puzzles provided they are framed in a novel way (it may have already seen some puzzles, but I doubt it can extend that knowledge to a puzzle with different words)
- quotemstr 6y ago
  This version of GPT can add, subtract, multiply, and divide without ever having been taught to do these things things. Yes, it can reason.
oehtXRwMkIs 6y ago
For most of human history text has not existed. I think you are conflating language with written language which is quite common in the post-Gutenberg age. I think sentience like most things is a spectrum, so I'm not really sure what you mean by sentient, but I would argue that for most people the bar for sentience is much higher than text prediction. The Chinese room is only one aspect of our minds, and we don't even know what consciousness is.
- quotemstr 6y ago
  I have to disagree with this perspective and reiterate my original position: language is sentience. And to be fair, reasonable people stake out positions on both sides of this debate: I'm not claiming that the alternative proposition is somehow unreasonable. It's a legitimate subject of scholarly disagreement. Nevertheless, I'm still firm on language. Why? Because all complexity is ultimately about symbolic manipulation of terms representing the process of manipulation itself. ("Godel, Escher, Bach" is a fantastic exploration of this concept.) How can you manipulate concepts without assigning terms to their parts? That's what language is. The question I like to ask is this: are there any ideas that you cannot express using language? No? Then how is thought distinct from language? Yes, people (myself included) experience a "tip of the tongue" experience where you feel like you have an idea you can't just yet express. But maybe this experience is what reason feels like. Why should idea formation take only one "clock cycle" in the brain? Why should we be unaware of the process? I think this feeling of having an idea yet being unable to formulate it is just the neural equivalent of a CPU pipeline stall. It's not evidence that we can have ideas without language: it's evidence that ideas sometime take a little while to gel.
  - oehtXRwMkIs 6y ago
    I think you're mixing up sentience/consciousness/intelligence. Many animals are sentient for example but as far as we know, they don't really have language. But I think I get what you're getting at, which I believe is "human level intelligence requires langauge". I think that's a reasonable take. But you said "sentience", and you said "is", which makes your position difficult to agree with.
pastrami_panda 6y ago
> I don't believe that there's any experience of consciousness distinct from the use of language. To me this indicates a very narrow view of consciousness. Consider for a moment the quiet consciousness of the cerebellum for example. I like the way David F. Wallace put it: 'Both flesh and not'. There's an astounding amount of consciousness that is not bound by language. One can even argue that language might hinder those forms of consciousness from even arising.
- quotemstr 6y ago
  If you believe in consciousness divorced from flesh, you're in strong intellectual company, but that's not a path I can go down. Metaphysically, I just can't accept the idea that the mind is anything but the computation performed by the brain.
  - henearkr 6y ago
    I rather agree with you, but the interaction of the mind with the physical reality is also extremely important to shape it. GPT3 has no interaction with a physical world. Any formal system that cannot interact with something outside of itself will be intrinsically limited, for one thing by Gödel incompleteness theorem.
    - visarga 6y ago
      I agree about the need for an environment. The difference between GPT-3 and an agent in an environment is that GPT-3 only saw tons of static text, while an agent can design an action (an experiment), act it out and observe the results, drawing conclusions. Thus it can act in a way similar to the scientific method.
  - pastrami_panda 6y ago
    In the book the meaning of the quote is more to the effect of how truly great athletes can perform on the verge of what spectators would consider inhuman or possible. I'd be hard-pressed to believe that language and syntax would be responsible for these kinds of actions and flow. I'd argue that getting into such a state is not possible while the mind is caught up in the language of things rather than the experience itself, and reacting to it directly. This is what I meant by the quiet consciousness, devoid of language or syntax.
jodrellblank 6y ago
> I don't believe that there's any experience of consciousness distinct from the use of language. Not sure there's one I can communicate to you, but I'm perfectly capable of forgetting the word for something and still knowing unambiguously yet wordlessly what it is, that's an experience. Catching a ball? Running? Experiencing emotions from wordless music? Viewing scenery? Engaging with a computer game? How are they not conscious experiences?
- jengleton 6y ago
  Also interesting to note that a good portion of people lack an internal monologue. This interview made the rounds a while back: https://www.youtube.com/watch?v=u69YSh-cFXY https://www.youtube.com/watch?v=u69YSh-cFXY
visarga 6y ago
> I don't believe that there's any experience of consciousness distinct from the use of language. What is the role of the body in consciousness, then?

visarga 6y ago

GPT-3 has flexible goals too. It can learn a new task and do it in the same step. What GPT3 doesn't have is a body.

solidasparagus 6y ago
No, you can teach it a new task or to do multiple tasks, but it will never be able to independently identify what a new task might be and learn to do it. An important distinction when talking about AGI.