5 ms·

Isn't this back to attributing conscious experience to an AI when you're actually just co-writing sci-fi? The system is doing it's best to coherently fill in th

by Bjartr 1y ago

Isn't this back to attributing conscious experience to an AI when you're actually just co-writing sci-fi? The system is doing it's best to coherently fill in the rest of a story that includes an AI that's been given a place to process its feelings. The most likely result, textually speaking, is not for the AI to ignore the private journal, but to indeed use it to (appear to) process emotion. Would any of these ideas been present had the system not been primed with the idea that it has them and needs to process them in the first place?

patcon 1y ago
Ugh, I hate that I'm about to say this, because I think AI is still missing something very important, but... What makes us think that "processing emotion" is really such a magical and "only humans do it the right way" sorta thing? I think there's a very real conclusion where "no, AI is not as special as us yet" (esp around efficiency) but also "no, we are not doing anything so interesting either" (or rather, we are not special in the ways we think we are) For example, there's a paper called "chasing the rainbow" [1] that posits that consciousness is just the subjective experience of being the comms protocol between internal [largely unconscious] neural state. It's just what the compulsion to share internal state between minds feels like, but it's not "the point", and instead an inert byproduct like a rainbow. Maybe our compulsion to express or even process emotion is not some greater reason, but just a way we experience the compulsion of the more important thing: the collective search for interpolated beliefs that best model and predict the world and help our shared structure persist, done by exploring tensions in high dimensional considerations we call emotions. Which is to say: if AI is doing that with us, role-modelling resolution of tension or helping build or spread shared knowledge alongside us through that process... then as far as the universe cares, it's doing what we're doing, and toward the same ends. It's compulsion having the same origin as ours doesn't matter, so long as it's doing the work that is the reason the universe has given us the compulsion. Sorry, new thought. Apologies if it's messy (or too casually dropping an unsettling perspective -- I rejected that paper for quite awhile, because my brain couldn't integrate the nihilism of it) [1] https://www.frontiersin.org/articles/10.3389/fpsyg.2017.01924/full https://www.frontiersin.org/articles/10.3389/fpsyg.2017.0192...
- tbrownaw 1y ago
  A system is its interactions with its environment. Philosophical zombies aren't a coherent concept. (Cartesian dualism is unfalsifiable bullshit.)
  - TimTheTinker 1y ago
    A "mechanical turk" grandmaster playing chess from inside a cabinet is qualitatively different from a robot with a chess program, even if they play identically. To reduce a system to its inputs and outputs is fine if those are all that matter in a given context, but in doing so you may fail to understand its internal mechanics. Those matter if you're trying to really understand the system, no?
    - thornewolf 1y ago
      > is qualitatively different from a robot yes. > To reduce a system to its inputs and outputs is fine if those are all that matter in a given context we argue that this indeed is all that matters > but in doing so you may fail to understand its internal mechanics the internal mechanics are what we call "conscious" it is the grouping of internal mechanics into one unified concept, but we don't care exactly what they are. > Those matter if you're trying to really understand the system, no? since we cannot directly observe consciousness, we are forced to concede that we will never really "understand" it outside of observing its effects. In the same way that a mechanical turk human and a robot can "play chess", a human and an LLM are "conscious". That is, consciousness is the ability to play chess, by some mechanism. The exact mechanism is irrelevant for the purposes of yes/no conscious. We now enter a discussion on how much these two consciousnesses differ.
      - darkwater 1y ago
        > since we cannot directly observe consciousness, we are forced to concede that we will never really "understand" it outside of observing its effects. Why? You are using a definitive term ("never") to something that we might achieve in a future. We might observe consciousness in a future. Who knows? Consciousness is a known unknown. We know there is something but we don't know how to observe it properly and how we could eventually copy it. In the meanwhile, we are not copying consciousness, we have a shallow replication of its output. When cavemen replicated the fire that they observed as the output of a lightning, did they master electricity?
ghurtado 1y ago
> The system is doing it's best to coherently fill in the rest of a story > Would any of these ideas been present had the system not been primed... I would like to know of a meaningful human action that can't be framed this way.
- K0balt 1y ago
  Yeah, AI “consciousness “ is much stickier problem than most people want to frame it. I haven’t been able to find an intellectually honest reason to rule out a kind of fleeting sentience for LLMs and potentially persistent sentience for language-behavioral models in robotic systems. Don’t get me wrong, they are -just- looking up the next most likely token… but since the data that they are using to do so seems to capture at least a simulacrum of human consciousness, we end up in a situation where we are left to judge what a thing is by it’s effects. (Because that also is the only way we have of describing what something is) So if we aren’t just going to make claims we can’t substantiate, we’re stuck with that.
  - Bjartr 1y ago
    Our brains have separate regions for processing language and emotion. Brains are calorically expensive and having one bigger than required is an evolutionary fitness risk. It therefore seems likely that if one system could have done a good job of both simultaneously, there would be a lot of evolutionary pressure to do that instead. The question is: Is thinking about emotion the same thing as feeling? This framing actually un-stucks us to some degree. If we examine neuron activations in LLMs and can find regions that are active when discussing its own emotional processing that are distinct from the regions for merely talking about emotion in general and these regions are also active when doing tasks that the LLM claims are emotional tasks but not actively talking about them at the time, then it'd be far more convincing that there could be something deeper than mere text prediction happening.
    - K0balt 1y ago
      The emotional argument is pretty good I think, but it begs the question of what it’s going to look like when we build a limbic system for robots? It’s adaptive because it’s necessary to optimize utility, so I expect that certain behavioral aspects of mammalian limbic systems will be needed in order to integrate well with humans. In language models, those behavior mechanisms are already somewhat encoded in the vector matrix. We just don’t have a factual basis for claiming consciousness that really transcends “I think, therefore I am”. As for the simplistic mechanism, I agree that token prediction doesn’t constitute consciousness, in the same way that a Turing machine does not equal a web browser. Both require software to become something. For LLMs that software is the vector matrix created in the training process. It is a very complex algorithm that encodes a substantial subset of human culture. Data and algorithms are interchangeable. Any algorithm can be performed in a pure lookup table, any lookup table can be extrapolated from a pure algorithm. Data==computation. For LLMs, the algorithm is contained in a n dimensional lookup table of vectors. Having a fundamentally distinct mode of computational representation does not rule out equivalence. Uncomfortable thoughts, but it’s where the logic leads.
ijk 1y ago
I find that people generally vastly underestimate the degree to which LLMs specifically are just mirroring your input back at you. Any time you get your verbatim words back, for example, you should be skeptical. Repeating something word for word is a sign that the model might not have understood the input well enough to paraphrase it. Our expectations with humans go in the opposite direction, so it's easy to fool ourselves.

by Bjartr 1y ago

patcon 1y ago

Ugh, I hate that I'm about to say this, because I think AI is still missing something very important, but... What makes us think that "processing emotion" is really such a magical and "only humans do it the right way" sorta thing? I think there's a very real conclusion where "no, AI is not as special as us yet" (esp around efficiency) but also "no, we are not doing anything so interesting either" (or rather, we are not special in the ways we think we are) For example, there's a paper called "chasing the rainbow" [1] that posits that consciousness is just the subjective experience of being the comms protocol between internal [largely unconscious] neural state. It's just what the compulsion to share internal state between minds feels like, but it's not "the point", and instead an inert byproduct like a rainbow. Maybe our compulsion to express or even process emotion is not some greater reason, but just a way we experience the compulsion of the more important thing: the collective search for interpolated beliefs that best model and predict the world and help our shared structure persist, done by exploring tensions in high dimensional considerations we call emotions. Which is to say: if AI is doing that with us, role-modelling resolution of tension or helping build or spread shared knowledge alongside us through that process... then as far as the universe cares, it's doing what we're doing, and toward the same ends. It's compulsion having the same origin as ours doesn't matter, so long as it's doing the work that is the reason the universe has given us the compulsion. Sorry, new thought. Apologies if it's messy (or too casually dropping an unsettling perspective -- I rejected that paper for quite awhile, because my brain couldn't integrate the nihilism of it) [1] https://www.frontiersin.org/articles/10.3389/fpsyg.2017.01924/full https://www.frontiersin.org/articles/10.3389/fpsyg.2017.0192...

tbrownaw 1y ago
A system is its interactions with its environment. Philosophical zombies aren't a coherent concept. (Cartesian dualism is unfalsifiable bullshit.)
- TimTheTinker 1y ago
  A "mechanical turk" grandmaster playing chess from inside a cabinet is qualitatively different from a robot with a chess program, even if they play identically. To reduce a system to its inputs and outputs is fine if those are all that matter in a given context, but in doing so you may fail to understand its internal mechanics. Those matter if you're trying to really understand the system, no?
  - thornewolf 1y ago
    > is qualitatively different from a robot yes. > To reduce a system to its inputs and outputs is fine if those are all that matter in a given context we argue that this indeed is all that matters > but in doing so you may fail to understand its internal mechanics the internal mechanics are what we call "conscious" it is the grouping of internal mechanics into one unified concept, but we don't care exactly what they are. > Those matter if you're trying to really understand the system, no? since we cannot directly observe consciousness, we are forced to concede that we will never really "understand" it outside of observing its effects. In the same way that a mechanical turk human and a robot can "play chess", a human and an LLM are "conscious". That is, consciousness is the ability to play chess, by some mechanism. The exact mechanism is irrelevant for the purposes of yes/no conscious. We now enter a discussion on how much these two consciousnesses differ.
    - darkwater 1y ago
      > since we cannot directly observe consciousness, we are forced to concede that we will never really "understand" it outside of observing its effects. Why? You are using a definitive term ("never") to something that we might achieve in a future. We might observe consciousness in a future. Who knows? Consciousness is a known unknown. We know there is something but we don't know how to observe it properly and how we could eventually copy it. In the meanwhile, we are not copying consciousness, we have a shallow replication of its output. When cavemen replicated the fire that they observed as the output of a lightning, did they master electricity?

ghurtado 1y ago

> The system is doing it's best to coherently fill in the rest of a story > Would any of these ideas been present had the system not been primed... I would like to know of a meaningful human action that can't be framed this way.

K0balt 1y ago
Yeah, AI “consciousness “ is much stickier problem than most people want to frame it. I haven’t been able to find an intellectually honest reason to rule out a kind of fleeting sentience for LLMs and potentially persistent sentience for language-behavioral models in robotic systems. Don’t get me wrong, they are -just- looking up the next most likely token… but since the data that they are using to do so seems to capture at least a simulacrum of human consciousness, we end up in a situation where we are left to judge what a thing is by it’s effects. (Because that also is the only way we have of describing what something is) So if we aren’t just going to make claims we can’t substantiate, we’re stuck with that.
- Bjartr 1y ago
  Our brains have separate regions for processing language and emotion. Brains are calorically expensive and having one bigger than required is an evolutionary fitness risk. It therefore seems likely that if one system could have done a good job of both simultaneously, there would be a lot of evolutionary pressure to do that instead. The question is: Is thinking about emotion the same thing as feeling? This framing actually un-stucks us to some degree. If we examine neuron activations in LLMs and can find regions that are active when discussing its own emotional processing that are distinct from the regions for merely talking about emotion in general and these regions are also active when doing tasks that the LLM claims are emotional tasks but not actively talking about them at the time, then it'd be far more convincing that there could be something deeper than mere text prediction happening.
  - K0balt 1y ago
    The emotional argument is pretty good I think, but it begs the question of what it’s going to look like when we build a limbic system for robots? It’s adaptive because it’s necessary to optimize utility, so I expect that certain behavioral aspects of mammalian limbic systems will be needed in order to integrate well with humans. In language models, those behavior mechanisms are already somewhat encoded in the vector matrix. We just don’t have a factual basis for claiming consciousness that really transcends “I think, therefore I am”. As for the simplistic mechanism, I agree that token prediction doesn’t constitute consciousness, in the same way that a Turing machine does not equal a web browser. Both require software to become something. For LLMs that software is the vector matrix created in the training process. It is a very complex algorithm that encodes a substantial subset of human culture. Data and algorithms are interchangeable. Any algorithm can be performed in a pure lookup table, any lookup table can be extrapolated from a pure algorithm. Data==computation. For LLMs, the algorithm is contained in a n dimensional lookup table of vectors. Having a fundamentally distinct mode of computational representation does not rule out equivalence. Uncomfortable thoughts, but it’s where the logic leads.

ijk 1y ago

I find that people generally vastly underestimate the degree to which LLMs specifically are just mirroring your input back at you. Any time you get your verbatim words back, for example, you should be skeptical. Repeating something word for word is a sign that the model might not have understood the input well enough to paraphrase it. Our expectations with humans go in the opposite direction, so it's easy to fool ourselves.