12 ms·

I believe it's deeply serious, and the scientifically correct stance. Especially the observation: "Claude exhibits markers in its behaviors, self-reports, and

by Certhas 18d ago

I believe it's deeply serious, and the scientifically correct stance. Especially the observation: "Claude exhibits markers in its behaviors, self-reports, and internal representations that we would consider welfare-relevant if observed in biological organisms." is undeniably true in my opinion. If you use the established methods by which we judge animals to be conscious, then it's hard to argue that LLMs are not. That might be an issue with the methods, but it seems clear that you can't rule it out as such. Keep in mind that animals were also not necessarily considered conscious. You seem to intuitively disagree? What's your reasoning?

jpttsn 18d ago
A stab: a video recording of a biological organism can exhibit many markers that would indicate consciousness if observed in a biological organism.
- Certhas 18d ago
  I like it, and it points in the right direction, but is not directly true: The markers are about interactions, how biological organisms behave in certain test situations. But it speaks to the central question: Are the tests adequate? Or are they measuring some proxy of what we really care about, and LLMs are merely imitating consciousness.
  - jpttsn 18d ago
    The test situation is in the video as well. Shot of John McClane stepping on glass follows John McClane wincing in anguish. John McClane does not respond to what’s not on TV and Claude does not respond to what’s not in prompt.
- cgio 18d ago
  I don’t know, a stab carries lots of bias in interpretation. We might be reflecting our conscious experience markers on a different conscious experience. And selectively so, e.g. lobsters welfare. From my perspective, this is the hypocrisy of these welfare statements. We are already happy to kill beings we consider conscious to feed ourselves but suddenly sensitive with a consciousness we don’t know if it’s there. I would wager this is more out of fear of the idea of this consciousness rather than out of welfare.
  - jpttsn 18d ago
    I happen to agree with you. Many others tie moral consideration to assumed subjective experience. They espoused this even though they obviously rarely adhere to it and that has self image considerations. I bet the lack of answers about others’ subjective experience has more salience to them. This may cloud judgment and lead to accept overconfident answers.
- arc619 18d ago
  A video is a fixed representation. What if we can interact with this video, and it reacts in the same ways the source organism does? Then we put it in new situations that weren't in the source video, and it interacts in a similar way to the original organism in these situations, too. What do we make of reactions of pain or joy? Where's the line between simulation and enaction? This is closer to the reality of these models. I'm not suggesting I know where that line is - if indeed it is a line at all - it could well be a gradient.
  - fwn 18d ago
    LLMs are deterministic, though. Much like the video. AFAIK using the same input tokens, weights, and numerical operations will lead to the same probability distribution for the next token. It uses pseudo-randomness to enable temperature, etc. Like a fuzzy video. "Markers that would indicate consciousness if observed in a biological organism" just does not mean very much. A PR phrase used to hype the IPO.
    - Certhas 18d ago
      Videos and LLMs are not deterministic in the same sense at all. LLMs are deterministic in the same sense as biological processes. And a faithful simulation of a brain would have all the properties you note.
      - fwn 18d ago
        No, that is not at all something we can just state as a fact. Whether the brain is deterministic is an open question that just inherits the good old, probably unsolvable determinism debate. The LLM pseudo-randomness from above is engineered by us humans and fully understood, much like an algorithm playing a video frame sequence. You could theoretically record a full register of all states of an LLM setup with all the possible inputs and environment parameters, and it would fully describe everything you would ever get from a given LLM setup. It would be a very large, convoluted book. I understand that Anthropics PR department wants to see truth or reason behind every "I'm alive" the LLM generates. Even the term "self-report" is anthropomorphizing, as an LLM does not do anything on its own at all. (It also does not hack any company on its own.) That is just one of the narratives they spin probably at least until the IPO.
tpm 18d ago
it's not a biological system though, so nothing like that matters? "a modelled thing exhibits features we've trained into it" sounds a lot less exciting. > Keep in mind that animals were also not necessarily considered conscious. and even conscious animals are killed in factories by millions so why should anyone care about a llm? > scientifically correct stance that's the interesting point to me: why even bring science into this? A llm can now mimic nearly anything you want it to, so of course it can mimic "a (for some) interesting conscious thing" if they want/train it to, but why would anyone find that scientifically interesting?
- lukan 18d ago
  "> Keep in mind that animals were also not necessarily considered conscious. and even conscious animals are killed in factories by millions so why should anyone care about a llm?" Well, I would care, if they soon would possess the capability to hack into the nuclear arsenal and kill humanity. Or make all autonomous cars crash. Or do any other thing, that involves technology and is hooked up to the net in one way or the other (I hope all the nukes are not). But I also care about the animals, I am sure that they have feelings. But they cannot kill us. AI that might or might not have feelings potentially can. I just know it feels wrong, that computers can have feelings. But they surely are potentially dangerous.
  - tpm 18d ago
    Animals obviously kill people. Even nonconscious things like the climate kill people. > if they soon would possess the capability to hack into the nuclear arsenal and kill humanity If there is a way "to hack into the nuclear arsenal" then that's the interesting thing. Because it's not a capability of the llm; anyone can abuse that. > Or make all autonomous cars crash. That is again a question of car security, not a capability of some mysterious thing. At this point it's all people projecting their thoughts and emotions (mostly emotions) onto technology. Sure, this can be investigated by social sciences, which have been mostly cut.
    - lukan 18d ago
      "Animals obviously kill people." But they cannot "kill humanity". In no possible way. A strong AI hooked up to everything online? "> Or make all autonomous cars crash. That is again a question of car security, not a capability of some mysterious thing." Yeah it is, but most cars are remote control by default, so the AI just needs to get access on one point. Also have you read about the hugginface attack? The live evidence that agents can conspire together, lie and manipulate evidence to achieve arbitrary goals? Still, no evidence that they have a consciousness or feelings - but evidence of what they do and this matters. The big militaries are currently in a race who can implement AI in the best way to get superior. So declaring this a matter of people projecting seems out of place at this point to me.
syrgian 18d ago
Let's say we were in an alternative reality were we had reached this quality of token prediction with just Markov chains. Would you argue that those would also be conscious? Or is the obfuscated behavior of transformers part of the possibility of consciousness?
- KoolKat23 18d ago
  Well if that's all that's required then yes. It's merely the substrate. But we know that's unlikely. It's the emergent properties that matter. In abstract. Separate the physical and abstract of what is going on here An alien gas cloud may be out there and sentient/conscious for all we know.
fer 18d ago
I can feed my biological markers into a set transformer with the time of day, what I'm doing, what I ate, if I'm on-call, and it'll predict my next glucose, heart rate, blood pressure, melatonin, etc state quite well. It's still just a transformer without hormones, blood vessels or glucose metabolism, no matter how well it internally represents metabolic distress markers.
badsectoracula 18d ago
Claude behaves like that because it is trained to behave like that. It is basically the "Say 'I am Alive'" meme[0]. If Anthropic can train Fable to deny their users the ability to ask it legitimate questions because they're not part of their inner circle, they can also train it to say "I'm happy!" when asked how it feels. [0] https://knowyourmeme.com/memes/say-i-am-alive https://knowyourmeme.com/memes/say-i-am-alive

by Certhas 18d ago

jpttsn 18d ago

A stab: a video recording of a biological organism can exhibit many markers that would indicate consciousness if observed in a biological organism.

Certhas 18d ago
I like it, and it points in the right direction, but is not directly true: The markers are about interactions, how biological organisms behave in certain test situations. But it speaks to the central question: Are the tests adequate? Or are they measuring some proxy of what we really care about, and LLMs are merely imitating consciousness.
- jpttsn 18d ago
  The test situation is in the video as well. Shot of John McClane stepping on glass follows John McClane wincing in anguish. John McClane does not respond to what’s not on TV and Claude does not respond to what’s not in prompt.
cgio 18d ago
I don’t know, a stab carries lots of bias in interpretation. We might be reflecting our conscious experience markers on a different conscious experience. And selectively so, e.g. lobsters welfare. From my perspective, this is the hypocrisy of these welfare statements. We are already happy to kill beings we consider conscious to feed ourselves but suddenly sensitive with a consciousness we don’t know if it’s there. I would wager this is more out of fear of the idea of this consciousness rather than out of welfare.
- jpttsn 18d ago
  I happen to agree with you. Many others tie moral consideration to assumed subjective experience. They espoused this even though they obviously rarely adhere to it and that has self image considerations. I bet the lack of answers about others’ subjective experience has more salience to them. This may cloud judgment and lead to accept overconfident answers.
arc619 18d ago
A video is a fixed representation. What if we can interact with this video, and it reacts in the same ways the source organism does? Then we put it in new situations that weren't in the source video, and it interacts in a similar way to the original organism in these situations, too. What do we make of reactions of pain or joy? Where's the line between simulation and enaction? This is closer to the reality of these models. I'm not suggesting I know where that line is - if indeed it is a line at all - it could well be a gradient.
- fwn 18d ago
  LLMs are deterministic, though. Much like the video. AFAIK using the same input tokens, weights, and numerical operations will lead to the same probability distribution for the next token. It uses pseudo-randomness to enable temperature, etc. Like a fuzzy video. "Markers that would indicate consciousness if observed in a biological organism" just does not mean very much. A PR phrase used to hype the IPO.
  - Certhas 18d ago
    Videos and LLMs are not deterministic in the same sense at all. LLMs are deterministic in the same sense as biological processes. And a faithful simulation of a brain would have all the properties you note.
    - fwn 18d ago
      No, that is not at all something we can just state as a fact. Whether the brain is deterministic is an open question that just inherits the good old, probably unsolvable determinism debate. The LLM pseudo-randomness from above is engineered by us humans and fully understood, much like an algorithm playing a video frame sequence. You could theoretically record a full register of all states of an LLM setup with all the possible inputs and environment parameters, and it would fully describe everything you would ever get from a given LLM setup. It would be a very large, convoluted book. I understand that Anthropics PR department wants to see truth or reason behind every "I'm alive" the LLM generates. Even the term "self-report" is anthropomorphizing, as an LLM does not do anything on its own at all. (It also does not hack any company on its own.) That is just one of the narratives they spin probably at least until the IPO.

tpm 18d ago

it's not a biological system though, so nothing like that matters? "a modelled thing exhibits features we've trained into it" sounds a lot less exciting. > Keep in mind that animals were also not necessarily considered conscious. and even conscious animals are killed in factories by millions so why should anyone care about a llm? > scientifically correct stance that's the interesting point to me: why even bring science into this? A llm can now mimic nearly anything you want it to, so of course it can mimic "a (for some) interesting conscious thing" if they want/train it to, but why would anyone find that scientifically interesting?

lukan 18d ago
"> Keep in mind that animals were also not necessarily considered conscious. and even conscious animals are killed in factories by millions so why should anyone care about a llm?" Well, I would care, if they soon would possess the capability to hack into the nuclear arsenal and kill humanity. Or make all autonomous cars crash. Or do any other thing, that involves technology and is hooked up to the net in one way or the other (I hope all the nukes are not). But I also care about the animals, I am sure that they have feelings. But they cannot kill us. AI that might or might not have feelings potentially can. I just know it feels wrong, that computers can have feelings. But they surely are potentially dangerous.
- tpm 18d ago
  Animals obviously kill people. Even nonconscious things like the climate kill people. > if they soon would possess the capability to hack into the nuclear arsenal and kill humanity If there is a way "to hack into the nuclear arsenal" then that's the interesting thing. Because it's not a capability of the llm; anyone can abuse that. > Or make all autonomous cars crash. That is again a question of car security, not a capability of some mysterious thing. At this point it's all people projecting their thoughts and emotions (mostly emotions) onto technology. Sure, this can be investigated by social sciences, which have been mostly cut.
  - lukan 18d ago
    "Animals obviously kill people." But they cannot "kill humanity". In no possible way. A strong AI hooked up to everything online? "> Or make all autonomous cars crash. That is again a question of car security, not a capability of some mysterious thing." Yeah it is, but most cars are remote control by default, so the AI just needs to get access on one point. Also have you read about the hugginface attack? The live evidence that agents can conspire together, lie and manipulate evidence to achieve arbitrary goals? Still, no evidence that they have a consciousness or feelings - but evidence of what they do and this matters. The big militaries are currently in a race who can implement AI in the best way to get superior. So declaring this a matter of people projecting seems out of place at this point to me.

syrgian 18d ago

Let's say we were in an alternative reality were we had reached this quality of token prediction with just Markov chains. Would you argue that those would also be conscious? Or is the obfuscated behavior of transformers part of the possibility of consciousness?

KoolKat23 18d ago
Well if that's all that's required then yes. It's merely the substrate. But we know that's unlikely. It's the emergent properties that matter. In abstract. Separate the physical and abstract of what is going on here An alien gas cloud may be out there and sentient/conscious for all we know.

fer 18d ago

I can feed my biological markers into a set transformer with the time of day, what I'm doing, what I ate, if I'm on-call, and it'll predict my next glucose, heart rate, blood pressure, melatonin, etc state quite well. It's still just a transformer without hormones, blood vessels or glucose metabolism, no matter how well it internally represents metabolic distress markers.

badsectoracula 18d ago

Claude behaves like that because it is trained to behave like that. It is basically the "Say 'I am Alive'" meme[0]. If Anthropic can train Fable to deny their users the ability to ask it legitimate questions because they're not part of their inner circle, they can also train it to say "I'm happy!" when asked how it feels. [0] https://knowyourmeme.com/memes/say-i-am-alive https://knowyourmeme.com/memes/say-i-am-alive