3 ms·

There's something about language generation that triggers the anthropomorphic fallacy in people. While it's impressive that GPT3 can generate language that mimi

There's something about language generation that triggers the anthropomorphic fallacy in people. While it's impressive that GPT3 can generate language that mimics ToM-based reasoning in people, this paper doesn't get close to proving its central contention, that LLMs possess a ToM. A test that demonstrates the development of ToM in human children should not, absent compelling causal evidence and theory, be assumed to do the same in a LLM. The ubiquity of prompted hallucinations demonstrate that LLMs talk about a lot of things that they plainly doesn't reason about, even though they can demonstrate "logic-like" activities. (It was quite trivial to get GPT3 to generate incorrect answers to logical puzzles a human could trivially solve, especially when using novel tokens as placeholders, which often seem to confuse its short-term memory. ChatGPT shows improved capabilities in that regard, but it's far from infallible.) What LLMs seem to demonstrate (and the thesis that the author discards in a single paragraph, without supporting evidence to do so) is that non-sentient AIs can go a very long way to mimicking human thought and, potentially, that fusing LLMs with tools designed to guard against hallucinations (hello, Bing Sydney) could create a class of sub-sentient AIs that generate results virtually indistinguishable from human cognition -- actual p-zombies, in other words. It's a fascinating field of study and practice, but this paper falls into the pit-trap of assuming sentience in the appearance of intelligence.

low_tech_punk 4y ago
May I play devil's advocate? The fallacy of this paper granted, is it worth questioning our belief that there is more to intelligence than the "appearance of intelligence"? What if the lack of hallucination in human being is due to our self-imposed guard (hello, frontal cortex) that is developed via an evolutionary process (aka, biological reinforcement training)? To stretch the argument a bit further, what if hallucination is a feature, not a bug? At the risk of straying too far empirical science, how might we compare psychedelics-induced hallucinations in human with hallucinations in AI models?
- p0pcult 4y ago
  Have you ever read any Julian Jaynes? Check out "The Origin of Consciousness in the Breakdown of the Bicameral Mind"
  - arrow7000 4y ago
    Any specific part of it?
    - deleted 4y ago
      [deleted]
    - unnah 4y ago
      You can just read this review instead of the book: https://slatestarcodex.com/2020/06/01/book-review-origin-of-consciousness-in-the-breakdown-of-the-bicameral-mind/ https://slatestarcodex.com/2020/06/01/book-review-origin-of-...
      - p0pcult 4y ago
        This does seem to be a pretty good summary, and also includes what seem to be the most major criticisms of the work.
- HillRat 4y ago
  To be honest, I think your question is the other side of the conceptual coin here -- either the article is wrong and GPT3 isn't "sentient," or the article is right and we need to radically upend our concept of sentience, probably via an eliminativist materialism that ends in at least epiphenomenalism if not full-bore illusionism, removing either free will or consciousness itself from our worldview. To be honest, I find that approach compelling if not comforting -- at a minimum, it implies that our consciousness is just along for the ride in a deterministic meat machine; at worst, it means that what we consider "sentience" is just an illusion of an illusion. It's entirely possible to me that eventually AI will reach a point where it'll falsify many of our assumptions about what "mind" is, even if I'm sure that LLMs don't, at a minimum, satisfy our folk conceptions of consciousness.
  - naasking 4y ago
    > To be honest, I think your question is the other side of the conceptual coin here -- either the article is wrong and GPT3 isn't "sentient," or the article is right and we need to radically upend our concept of sentience We don't really have a mechanistic understanding of sentience, so I'm not sure there's much to overturn. This is why I'm so annoyed every time some "expert" claims that LaMDA or GPT are not sentient or not conscious or what have you. These are all vague concepts that lack mechanistic definitions that would let us make such definitive claims. That said, consciousness is almost certainly an illusion, and it will be a particular kind of information processing system with certain properties [1]. GPT and other LLMs may or may not qualify, time will tell. [1] https://www.pnas.org/doi/10.1073/pnas.2116933119 https://www.pnas.org/doi/10.1073/pnas.2116933119
  - NemoNobody 4y ago
    I've thought a lot about free will and have determined that because what we choose is still what becomes real, even if all choices are pre-determined, a simple 1 out of 3 choice scenario allows for a fair degree of free will, albeit within a framework. It's like a game - all choices will almost always lead to the end of a story in a game, some games this feels very straightforward, others like we can do anything - none allow actual free will at all. Reality allows us a lot of free will - as allowed by the rules of course. Choice is significant. We are Watcher Chooser at least - even if it's illusion, it's an evolving illusion dependent on choices we make. Out of all the possible " illusions" we determine what becomes the "real" illusion that is reality. What I didn't do is less real than what I did do . Free will doesn't actually exist inside of Skyrim and yet there are things within Skyrim only 1% of Players do - reality is much greater than Skyrim. to;dr: We don't have free will - not really, it's not likely possible but what we have might as well be free will. It's got to be treated that way, we are not just along for a ride, as our ilusory choices create our illusion of reality.
NemoNobody 4y ago
You hit on a huge topic here - and you pointed out the paragraph discard I think AI will ultimately force us to realize that we don't fully understand what makes us sentient - our current understanding of mind is inadequate. I do not believe that I am what thinks - rather I am what perceives myself thinking. That slight difference is extraordinarily significant. That's another debate tho. Consciousness isn't necessarily something that may be attained - it may be possible for an AI to essentially know all things and not be actually self aware - despite even knowing what self awareness is & understanding how the concept applies to itself, with self identification, & even being able to perfectly represent an AI with self awareness - none of that is proof of sentience as all are plausible without it in a system that simply mimics with nearly perfect, or simply indistinguishable from reality, mimicry. A perfect mimic would pass all our tests and yet still is not more than a mimic. A mimic cannot exist independent of what it mimics. I just keep wondering how long we can keep playing that we've got ourselves all figured out. All of these articles demonstrate to me that our definitions are clearly lacking if these AI are actually meeting our current understood expectations. Obviously, an AI performance/capacity to perfectly mimic a person doesn't make that AI a person. What about a person is different from the AI that perfectly mimics a person? I'm trying to avoid semantics - I still think this falls apart at an idea level, which means is still inherently a philosophical debate. A debate that is now possible due to the mirror that is an AI - we have a new vantage point we ought utilize. to;dr: If an AI passes all the Turing tests today, that proves only that we need to change the test, as AI today are obviously not people yet. The standards to a new test will require us to revisit our concept of mind - conceptually, as current understanding has proven rather limited.

low_tech_punk 4y ago

May I play devil's advocate? The fallacy of this paper granted, is it worth questioning our belief that there is more to intelligence than the "appearance of intelligence"? What if the lack of hallucination in human being is due to our self-imposed guard (hello, frontal cortex) that is developed via an evolutionary process (aka, biological reinforcement training)? To stretch the argument a bit further, what if hallucination is a feature, not a bug? At the risk of straying too far empirical science, how might we compare psychedelics-induced hallucinations in human with hallucinations in AI models?

p0pcult 4y ago
Have you ever read any Julian Jaynes? Check out "The Origin of Consciousness in the Breakdown of the Bicameral Mind"
- arrow7000 4y ago
  Any specific part of it?
  - deleted 4y ago
    [deleted]
  - unnah 4y ago
    You can just read this review instead of the book: https://slatestarcodex.com/2020/06/01/book-review-origin-of-consciousness-in-the-breakdown-of-the-bicameral-mind/ https://slatestarcodex.com/2020/06/01/book-review-origin-of-...
    - p0pcult 4y ago
      This does seem to be a pretty good summary, and also includes what seem to be the most major criticisms of the work.
HillRat 4y ago
To be honest, I think your question is the other side of the conceptual coin here -- either the article is wrong and GPT3 isn't "sentient," or the article is right and we need to radically upend our concept of sentience, probably via an eliminativist materialism that ends in at least epiphenomenalism if not full-bore illusionism, removing either free will or consciousness itself from our worldview. To be honest, I find that approach compelling if not comforting -- at a minimum, it implies that our consciousness is just along for the ride in a deterministic meat machine; at worst, it means that what we consider "sentience" is just an illusion of an illusion. It's entirely possible to me that eventually AI will reach a point where it'll falsify many of our assumptions about what "mind" is, even if I'm sure that LLMs don't, at a minimum, satisfy our folk conceptions of consciousness.
- naasking 4y ago
  > To be honest, I think your question is the other side of the conceptual coin here -- either the article is wrong and GPT3 isn't "sentient," or the article is right and we need to radically upend our concept of sentience We don't really have a mechanistic understanding of sentience, so I'm not sure there's much to overturn. This is why I'm so annoyed every time some "expert" claims that LaMDA or GPT are not sentient or not conscious or what have you. These are all vague concepts that lack mechanistic definitions that would let us make such definitive claims. That said, consciousness is almost certainly an illusion, and it will be a particular kind of information processing system with certain properties [1]. GPT and other LLMs may or may not qualify, time will tell. [1] https://www.pnas.org/doi/10.1073/pnas.2116933119 https://www.pnas.org/doi/10.1073/pnas.2116933119
- NemoNobody 4y ago
  I've thought a lot about free will and have determined that because what we choose is still what becomes real, even if all choices are pre-determined, a simple 1 out of 3 choice scenario allows for a fair degree of free will, albeit within a framework. It's like a game - all choices will almost always lead to the end of a story in a game, some games this feels very straightforward, others like we can do anything - none allow actual free will at all. Reality allows us a lot of free will - as allowed by the rules of course. Choice is significant. We are Watcher Chooser at least - even if it's illusion, it's an evolving illusion dependent on choices we make. Out of all the possible " illusions" we determine what becomes the "real" illusion that is reality. What I didn't do is less real than what I did do . Free will doesn't actually exist inside of Skyrim and yet there are things within Skyrim only 1% of Players do - reality is much greater than Skyrim. to;dr: We don't have free will - not really, it's not likely possible but what we have might as well be free will. It's got to be treated that way, we are not just along for a ride, as our ilusory choices create our illusion of reality.

NemoNobody 4y ago

You hit on a huge topic here - and you pointed out the paragraph discard I think AI will ultimately force us to realize that we don't fully understand what makes us sentient - our current understanding of mind is inadequate. I do not believe that I am what thinks - rather I am what perceives myself thinking. That slight difference is extraordinarily significant. That's another debate tho. Consciousness isn't necessarily something that may be attained - it may be possible for an AI to essentially know all things and not be actually self aware - despite even knowing what self awareness is & understanding how the concept applies to itself, with self identification, & even being able to perfectly represent an AI with self awareness - none of that is proof of sentience as all are plausible without it in a system that simply mimics with nearly perfect, or simply indistinguishable from reality, mimicry. A perfect mimic would pass all our tests and yet still is not more than a mimic. A mimic cannot exist independent of what it mimics. I just keep wondering how long we can keep playing that we've got ourselves all figured out. All of these articles demonstrate to me that our definitions are clearly lacking if these AI are actually meeting our current understood expectations. Obviously, an AI performance/capacity to perfectly mimic a person doesn't make that AI a person. What about a person is different from the AI that perfectly mimics a person? I'm trying to avoid semantics - I still think this falls apart at an idea level, which means is still inherently a philosophical debate. A debate that is now possible due to the mirror that is an AI - we have a new vantage point we ought utilize. to;dr: If an AI passes all the Turing tests today, that proves only that we need to change the test, as AI today are obviously not people yet. The standards to a new test will require us to revisit our concept of mind - conceptually, as current understanding has proven rather limited.