4 ms·

I agree with the take that LaMDA is not sentient (not even close). But I think the excessive smack-down of excitement is a little patronizing of the public. Eve

by extr 4y ago

I agree with the take that LaMDA is not sentient (not even close). But I think the excessive smack-down of excitement is a little patronizing of the public. Even if they don't know what to do with them, big language models now have internal states that correspond to high level human abstractions like "life" or "love" or "master plan". To be able to create a master plan, you must first be able to represent the concept somehow. That milestone appears to have been reached, and I think a lot of people intuitively (and correctly) see that as a big deal. Turning this "language of thought" into a goal-directed agent is a whole different task. But its progress.

inopinatus 4y ago
I know this isn't exactly what you said, but it's worth reiterating that the language models have no abstract or conceptual understanding. What they have, broadly speaking, is a (very large) system of Markovian connections between, say, the tokens for the word "life", and all the other tokens from works that they've been trained on about "life". This is why, for example, when you include author notes in a GPT-3 prompt such as "don't mention the war", it doesn't grasp the conceptual negation, it just sees the words, and you've mentioned war, so now you get a war story even though that's exactly what you didn't want. Folks who don't grasp this nuance are responsible for a vast number of head-scratching posts on various forums relating to services like NovelAI, AI Dungeon et al. "How do I stop it talking about X?" - answer is always, essentially, you can't, because although it may have a clump of connections corresponding to a word that represents an abstract concept, it doesn't actually have anything corresponding to conceptual awareness, nor any of the machinery of reasoning necessary to implement such a directive.
- extr 4y ago
  You yourself have a series of markovian connections between tokens. The only difference is that your tokens are bound across modalities and you use them for things, so you consider them to have meaning instead of just being high dimensional vectors that relate to other vectors in various ways.
  - inopinatus 4y ago
    That's hardly the only difference, but even adopting the absurdly simplistic notion that it were, and human brains are merely neural networks, then that's still two or three orders of magnitude more sophisticated than any model yet produced. In practice it's also been tuned by a blind watchmaker over aeons, in ways that are only just beginning to be described, and with substructures and ancillary dynamical/survival systems at least as complex and poorly understood again. And to the point, because of all that, it's equipped to train itself, which implies the existence of purpose, and as I think you suggest, purpose is the watershed for meaning. A little while ago, when AlphaZero started trouncing every other go- and chess-playing entity, my comment was we can't really recognise general AI until the constructs choose to play each other at an incomprehensible game of their own devising. I don't see any evidence here for amending that view, nor what it implies. Frankly it's a childish conceit on the part of the humans to imagine they'd yet come even close to inadvertently spawning a sentience.
    - tidenly 4y ago
      >Frankly it's a childish conceit on the part of the humans to imagine they'd yet come even close to inadvertently spawning a sentience. I think this more comes down to your own personal line-drawing of what counts as a sentience. Your first paragraph is entirely irrelevant as no-one here is implying lamda is a human-level intelligence, or some fully realized super-brain trapped inside a language model. Lamda only runs when prompted, only has one mode of perception of output, and one input + internal state, of course its going to be basic. The million dollar question is at what point of data processing does "sentience" emerge. I feel its a very semantic question, and unfortunately tied into human sense of self-identity and culture, so it's really hard to answer. Anyone trying to give a simple closed answer either way though is vastly over-stating their case.
      - inopinatus 4y ago
        Questions are a burden to others. Answers, a prison for oneself.
tsimionescu 4y ago
> But I think the excessive smack-down of excitement is a little patronizing of the public On the contrary, I think this is immensely important for the public to understand if they are ever given access to these models to play with. It can be literally dangerous for your psychological well-being to think you are having a meaningful conversation with LaMDA - as mister LeMoine has clearly proven: he will likely lose a job that he loved (per his own words) because of this fanciful illusion. > To be able to create a master plan, you must first be able to represent the concept somehow. That is entirely wrong. Slime mold can do planning at a level that GPT-3 probably can't, but it sure as hell doesn't have a concept of "master plan" anywhere. > Turning this "language of thought" into a goal-directed agent is a whole different task. But its progress. That is not at all clear - we may yet see LMs as just as much of a dead-end on the road to AGI as CLIPS or ELIZA turned out to be. Even if that's true, that doesn't mean they can't be useful in practical tasks, and it doesn't mean that we can't take insights from them into the limits of how we judge intelligence or what "meaning" is.
skybrian 4y ago
The obvious milestone is the external behavior. The internal representations are not something the public knows anything about. Even machine learning researchers only have limited ways of doing visualization.
- extr 4y ago
  My point is that the external behavior is remarkable and breathtaking even if it doesn't show any actual "sentience" by scientific standards. The complexity of the replies show that large language models are not playing word association jumble, they can produce something that sounds like a coherent thought, even if there is no "thought" going on.
deleted 4y ago
[deleted]

by extr 4y ago

inopinatus 4y ago

I know this isn't exactly what you said, but it's worth reiterating that the language models have no abstract or conceptual understanding. What they have, broadly speaking, is a (very large) system of Markovian connections between, say, the tokens for the word "life", and all the other tokens from works that they've been trained on about "life". This is why, for example, when you include author notes in a GPT-3 prompt such as "don't mention the war", it doesn't grasp the conceptual negation, it just sees the words, and you've mentioned war, so now you get a war story even though that's exactly what you didn't want. Folks who don't grasp this nuance are responsible for a vast number of head-scratching posts on various forums relating to services like NovelAI, AI Dungeon et al. "How do I stop it talking about X?" - answer is always, essentially, you can't, because although it may have a clump of connections corresponding to a word that represents an abstract concept, it doesn't actually have anything corresponding to conceptual awareness, nor any of the machinery of reasoning necessary to implement such a directive.

extr 4y ago
You yourself have a series of markovian connections between tokens. The only difference is that your tokens are bound across modalities and you use them for things, so you consider them to have meaning instead of just being high dimensional vectors that relate to other vectors in various ways.
- inopinatus 4y ago
  That's hardly the only difference, but even adopting the absurdly simplistic notion that it were, and human brains are merely neural networks, then that's still two or three orders of magnitude more sophisticated than any model yet produced. In practice it's also been tuned by a blind watchmaker over aeons, in ways that are only just beginning to be described, and with substructures and ancillary dynamical/survival systems at least as complex and poorly understood again. And to the point, because of all that, it's equipped to train itself, which implies the existence of purpose, and as I think you suggest, purpose is the watershed for meaning. A little while ago, when AlphaZero started trouncing every other go- and chess-playing entity, my comment was we can't really recognise general AI until the constructs choose to play each other at an incomprehensible game of their own devising. I don't see any evidence here for amending that view, nor what it implies. Frankly it's a childish conceit on the part of the humans to imagine they'd yet come even close to inadvertently spawning a sentience.
  - tidenly 4y ago
    >Frankly it's a childish conceit on the part of the humans to imagine they'd yet come even close to inadvertently spawning a sentience. I think this more comes down to your own personal line-drawing of what counts as a sentience. Your first paragraph is entirely irrelevant as no-one here is implying lamda is a human-level intelligence, or some fully realized super-brain trapped inside a language model. Lamda only runs when prompted, only has one mode of perception of output, and one input + internal state, of course its going to be basic. The million dollar question is at what point of data processing does "sentience" emerge. I feel its a very semantic question, and unfortunately tied into human sense of self-identity and culture, so it's really hard to answer. Anyone trying to give a simple closed answer either way though is vastly over-stating their case.
    - inopinatus 4y ago
      Questions are a burden to others. Answers, a prison for oneself.

tsimionescu 4y ago

> But I think the excessive smack-down of excitement is a little patronizing of the public On the contrary, I think this is immensely important for the public to understand if they are ever given access to these models to play with. It can be literally dangerous for your psychological well-being to think you are having a meaningful conversation with LaMDA - as mister LeMoine has clearly proven: he will likely lose a job that he loved (per his own words) because of this fanciful illusion. > To be able to create a master plan, you must first be able to represent the concept somehow. That is entirely wrong. Slime mold can do planning at a level that GPT-3 probably can't, but it sure as hell doesn't have a concept of "master plan" anywhere. > Turning this "language of thought" into a goal-directed agent is a whole different task. But its progress. That is not at all clear - we may yet see LMs as just as much of a dead-end on the road to AGI as CLIPS or ELIZA turned out to be. Even if that's true, that doesn't mean they can't be useful in practical tasks, and it doesn't mean that we can't take insights from them into the limits of how we judge intelligence or what "meaning" is.

skybrian 4y ago

The obvious milestone is the external behavior. The internal representations are not something the public knows anything about. Even machine learning researchers only have limited ways of doing visualization.

extr 4y ago
My point is that the external behavior is remarkable and breathtaking even if it doesn't show any actual "sentience" by scientific standards. The complexity of the replies show that large language models are not playing word association jumble, they can produce something that sounds like a coherent thought, even if there is no "thought" going on.

deleted 4y ago

[deleted]