5 ms·

Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions.

Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions. Is a car a body? Does an AI situated in a car therefore get to have desires and emotions? Is a taupe box with a webcam attached a body? (For that matter: Is a quadropelegic body a body? Do quadropelegics have desires and emotions? Obviously, yes and yes.) Why is a body necessary for the formation of desires and emotions? Why are desires and emotions necessary features for consciousness? Or here's one: If I'm not experiencing any particular emotion in a given moment, am I concious? I tend to think that emotions, at least, are mainly hormonal global triggers: they're more about physiology than actual consciousness. The whole thing, as a result, sounds like an effort to privilege biological intelligence, rather than a real foray into the issues.

lwarfield 4mo ago
>Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions. Damn, what a line! Another thing that bothered me with his baseline for consciousness was that it did not involve the ability to change one's self. A big part of being conscious in my mind is how one's experiences shape them, and how someone can shape themselves. LLMs completely lack this, their weights are static. An LLM isn't going to be molded by a bad breakup, or a relative passing away. An LLM isn't going to set up a routine to get stronger with training, nor smarter by reading up on a field.
- sdenton4 4mo ago
  The thing is... Interactive updates happen, just in a different way than it does for animal brains. The system is updated with new training data more-or-less constantly. Suppose OpenAI (or whoever) collects a week's worth of conversations with up-thumbs and down-thumbs, or rewritten continuations from human operators, then fine-tunes the current version of ChatGPT with that data. That's an interactive update, and learning from experience. It looks mostly nothing like what we humans do... But it does rhyme a little bit! We humans have mostly frozen weights (neurons), or else we would constantly be having to avoid forgetting how to walk+talk. We have a period of greater plasticity (youth!), and use sleep and dreams to perform 'deeper' updates than occur when we're awake: We tend to suck a bit at picking up new skills from zero, but improve rapidly with practice over days.
  - andoando 4mo ago
    Yup, you could definitely take it offline (sleep) every night, update it and turn it back on.
- felipeerias 4mo ago
  A stronger version of that argument is that LLMs are not intrinsically affected by the passage of time. Input stream comes in, input stream comes out. The LLM doesn't care whether this happens once a minute or once a year.
  - Muskwalker 4mo ago
    I don't normally experience the time interval between when my input streams shut down every night and reboot every morning either, though. Nothing prevents one from running an LLM whose harness has a clock and a while loop in it, and it would be weird if its mere lack were really so consequential to consciousness.
    - felipeerias 4mo ago
      Exactly, the clock is external to the model. Nothing prevents it from being faster or slower, or even running backwards, because it’s ultimately just another data point in the input stream to a computer function. Your brain and your whole body exist in time. Even when you are asleep, your body does not flicker out of existence and your brain actually continues working during that time.
gabrieledarrigo 4mo ago
> Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions. How can you have a subjective experience without a body? That's the point, in my opinion: your physical/chemical state (body) in a given moment is then translated into the higher abstraction of the emotion. An emotion that *you* feel, because you are self aware of what's happening. How can you be self aware without feeling? And how do you feel, without a body?
- Tadpole9181 4mo ago
  You need to draw that thinking out to it's natural conclusion, though. If I cut out your brain and stopped you from hearing or seeing or feeling - you would still be a conscious human being capable of thinking and awareness. If I hooked up electrodes to the hearing centers of your brain and force fed you dialog you perceive as speech (but is really a great deceiver), then responded in what you thought was speech (but are really just probes I use to convert your thoughts to text), that wouldn't suddenly be less real to you. It wouldn't devalue your sapience.
  - goatlover 4mo ago
    How do you know the brain separated from the rest of the nervous system and body would still be sapient, capable of thinking and awareness? There's an assumption you're making that the brain is all that's needed, but the nervous system extends throughout the body. One can argue sensory organ are part of the nervous system. Embodied cognition rejects this assumption. We didn't evolve as brains that were then put in bodies, we evolved as bodies with nervous systems.
    - gpderetta 4mo ago
      While dreaming, the brain synthetizes sensory experience while cutting down (or greatly suppressing) most of external stimuli. Yet it is still conscious.
      - deleted 4mo ago
pasquinelli 4mo ago
> Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. the subject is consciousness, not intelligence.
solid_fuel 4mo ago
I think you're missing his point. It's not about the hormones and physics of our bodies, and indeed he specifically allows for "either physical or virtual" bodies even in the block ~~~you~~~ grandparent quoted. The point here is not that it must have a body like ours, the point is that a conscious entity must have a boundary line between internal (the body) and external (everything else). A virtual sense organ can simply be an encoder or a web camera or a magnetometer, the specifics don't matter, what matters is that there are only a few bridges between the outer world and the inner world. Even if you want to call a tokenizer and autoencoder a "sense organ", LLMs are not embodied because there is no boundary line - there is no internal "thought" that is not directly descended from the prompt and there is no internal reasoning which is not immediately dumped into the external environment.
- cout 4mo ago
  Would it be sufficient to have a second stream of tokens that becomes the model's equivalent of "internal dialogue"? Would that satisfy the requirement of a boundary line? Related, is a human "thinking out loud" still thinking, even though the internal reasoning is "immediately dumped into the external environment"?
  - deleted 4mo ago
    [deleted]
datsci_est_2015 4mo ago
> If I'm not experiencing any particular emotion in a given moment, am I concious? No, you’re not. I think even baseline emotional responses to stimuli is table stakes for “consciousness”.
rvrs 4mo ago
>If I'm not experiencing any particular emotion in a given moment, am I concious? Genuine question: How are you so sure whether you are experiencing an emotion or not? Are you a master of everything inside your own head?

lwarfield 4mo ago

>Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions. Damn, what a line! Another thing that bothered me with his baseline for consciousness was that it did not involve the ability to change one's self. A big part of being conscious in my mind is how one's experiences shape them, and how someone can shape themselves. LLMs completely lack this, their weights are static. An LLM isn't going to be molded by a bad breakup, or a relative passing away. An LLM isn't going to set up a routine to get stronger with training, nor smarter by reading up on a field.

sdenton4 4mo ago
The thing is... Interactive updates happen, just in a different way than it does for animal brains. The system is updated with new training data more-or-less constantly. Suppose OpenAI (or whoever) collects a week's worth of conversations with up-thumbs and down-thumbs, or rewritten continuations from human operators, then fine-tunes the current version of ChatGPT with that data. That's an interactive update, and learning from experience. It looks mostly nothing like what we humans do... But it does rhyme a little bit! We humans have mostly frozen weights (neurons), or else we would constantly be having to avoid forgetting how to walk+talk. We have a period of greater plasticity (youth!), and use sleep and dreams to perform 'deeper' updates than occur when we're awake: We tend to suck a bit at picking up new skills from zero, but improve rapidly with practice over days.
- andoando 4mo ago
  Yup, you could definitely take it offline (sleep) every night, update it and turn it back on.
felipeerias 4mo ago
A stronger version of that argument is that LLMs are not intrinsically affected by the passage of time. Input stream comes in, input stream comes out. The LLM doesn't care whether this happens once a minute or once a year.
- Muskwalker 4mo ago
  I don't normally experience the time interval between when my input streams shut down every night and reboot every morning either, though. Nothing prevents one from running an LLM whose harness has a clock and a while loop in it, and it would be weird if its mere lack were really so consequential to consciousness.
  - felipeerias 4mo ago
    Exactly, the clock is external to the model. Nothing prevents it from being faster or slower, or even running backwards, because it’s ultimately just another data point in the input stream to a computer function. Your brain and your whole body exist in time. Even when you are asleep, your body does not flicker out of existence and your brain actually continues working during that time.

gabrieledarrigo 4mo ago

> Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions. How can you have a subjective experience without a body? That's the point, in my opinion: your physical/chemical state (body) in a given moment is then translated into the higher abstraction of the emotion. An emotion that *you* feel, because you are self aware of what's happening. How can you be self aware without feeling? And how do you feel, without a body?

Tadpole9181 4mo ago
You need to draw that thinking out to it's natural conclusion, though. If I cut out your brain and stopped you from hearing or seeing or feeling - you would still be a conscious human being capable of thinking and awareness. If I hooked up electrodes to the hearing centers of your brain and force fed you dialog you perceive as speech (but is really a great deceiver), then responded in what you thought was speech (but are really just probes I use to convert your thoughts to text), that wouldn't suddenly be less real to you. It wouldn't devalue your sapience.
- goatlover 4mo ago
  How do you know the brain separated from the rest of the nervous system and body would still be sapient, capable of thinking and awareness? There's an assumption you're making that the brain is all that's needed, but the nervous system extends throughout the body. One can argue sensory organ are part of the nervous system. Embodied cognition rejects this assumption. We didn't evolve as brains that were then put in bodies, we evolved as bodies with nervous systems.
  - gpderetta 4mo ago
    While dreaming, the brain synthetizes sensory experience while cutting down (or greatly suppressing) most of external stimuli. Yet it is still conscious.
    - deleted 4mo ago

pasquinelli 4mo ago

> Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. the subject is consciousness, not intelligence.

solid_fuel 4mo ago

I think you're missing his point. It's not about the hormones and physics of our bodies, and indeed he specifically allows for "either physical or virtual" bodies even in the block ~~~you~~~ grandparent quoted. The point here is not that it must have a body like ours, the point is that a conscious entity must have a boundary line between internal (the body) and external (everything else). A virtual sense organ can simply be an encoder or a web camera or a magnetometer, the specifics don't matter, what matters is that there are only a few bridges between the outer world and the inner world. Even if you want to call a tokenizer and autoencoder a "sense organ", LLMs are not embodied because there is no boundary line - there is no internal "thought" that is not directly descended from the prompt and there is no internal reasoning which is not immediately dumped into the external environment.

cout 4mo ago
Would it be sufficient to have a second stream of tokens that becomes the model's equivalent of "internal dialogue"? Would that satisfy the requirement of a boundary line? Related, is a human "thinking out loud" still thinking, even though the internal reasoning is "immediately dumped into the external environment"?
- deleted 4mo ago
  [deleted]

datsci_est_2015 4mo ago

> If I'm not experiencing any particular emotion in a given moment, am I concious? No, you’re not. I think even baseline emotional responses to stimuli is table stakes for “consciousness”.

rvrs 4mo ago

>If I'm not experiencing any particular emotion in a given moment, am I concious? Genuine question: How are you so sure whether you are experiencing an emotion or not? Are you a master of everything inside your own head?