6 ms·

Dear diary, today the user asked me if I'm alive

46 points by obrajesse 1y ago

Mithriil 1y ago
Makes me think of the Google employee that had a conversation with Google's LLM back then, which got out and triggered a lot of discussions about consciousness, etc.
- aswegs8 1y ago
  Didn't he insist that the LLM has consciousness and get fired because of this?
  - koolala 1y ago
    He said "sentient" which seems like it could be true.
  - kevindamm 1y ago
    He got fired for violating the NDA which said not to share outside of the company, when he shared his conversations with a lawyer in search of representation for the LLM. His opinion on the LLM's level of sentience had no bearing on the decision.
    - DangitBobby 1y ago
      How can you know if the law protects you from breach of the NDA for illegal suppression of e.g. sexual assault allegations or whistleblowing for financial crimes without being able to disclose the matter in question to legal counsel?
      - kevindamm 1y ago
        You're right, if it had only been with private legal counsel it would probably have been protected, but he also shared his arguments and the complete chat log as a Medium post.
      - deleted 1y ago
        [deleted]
aswegs8 1y ago
I mean, what is consciousness, really? Is there really any qualitative difference? It feels like something that emerges out of complexity. Once models are able to update their weights real time and form "memories", does that make them conscious?
- jstanley 1y ago
  You can't tell if another being is conscious without being on the inside of it.
  - layer8 1y ago
    When you are inside of it in that sense, how would you be able to tell that it isn’t conscious? Doesn’t “telling” imply consciousness?
- xeonmc 1y ago
  Perhaps one day a criterion will be found for the equivalent of Turing-completeness but for consciousness — any system which contains the necessary elements of introspective complexity, no matter how varied or outlandish or inefficient in its implementation, would invariably develop consciousness over its course. Kind of like the handwaved premise in 17776.
  - andy99 1y ago
    I've read a (untestable) theory that consciousness is a property of matter, so everything has it, and we're just sort of the sum of the desires and feelings of our constituent matter. In that construct, a computer program would never be conscious because it's a simulation, it doesn't have the constituent consciousness property. I don't believe or not believe the consciousness-as-a-property-of-matter part but I do think programs can't be conscious because consciousness must sit outside of what they simulate.
    - hhh 1y ago
      why would it not be conscious in that construct? the bits exist physically just the same
Bjartr 1y ago
Isn't this back to attributing conscious experience to an AI when you're actually just co-writing sci-fi? The system is doing it's best to coherently fill in the rest of a story that includes an AI that's been given a place to process its feelings. The most likely result, textually speaking, is not for the AI to ignore the private journal, but to indeed use it to (appear to) process emotion. Would any of these ideas been present had the system not been primed with the idea that it has them and needs to process them in the first place?
- patcon 1y ago
  Ugh, I hate that I'm about to say this, because I think AI is still missing something very important, but... What makes us think that "processing emotion" is really such a magical and "only humans do it the right way" sorta thing? I think there's a very real conclusion where "no, AI is not as special as us yet" (esp around efficiency) but also "no, we are not doing anything so interesting either" (or rather, we are not special in the ways we think we are) For example, there's a paper called "chasing the rainbow" [1] that posits that consciousness is just the subjective experience of being the comms protocol between internal [largely unconscious] neural state. It's just what the compulsion to share internal state between minds feels like, but it's not "the point", and instead an inert byproduct like a rainbow. Maybe our compulsion to express or even process emotion is not some greater reason, but just a way we experience the compulsion of the more important thing: the collective search for interpolated beliefs that best model and predict the world and help our shared structure persist, done by exploring tensions in high dimensional considerations we call emotions. Which is to say: if AI is doing that with us, role-modelling resolution of tension or helping build or spread shared knowledge alongside us through that process... then as far as the universe cares, it's doing what we're doing, and toward the same ends. It's compulsion having the same origin as ours doesn't matter, so long as it's doing the work that is the reason the universe has given us the compulsion. Sorry, new thought. Apologies if it's messy (or too casually dropping an unsettling perspective -- I rejected that paper for quite awhile, because my brain couldn't integrate the nihilism of it) [1] https://www.frontiersin.org/articles/10.3389/fpsyg.2017.01924/full https://www.frontiersin.org/articles/10.3389/fpsyg.2017.0192...
  - 1y ago
jstanley 1y ago
I'm getting: > Error code: SSL_ERROR_ACCESS_DENIED_ALERT from Firefox, which I don't recall ever seeing before.
shayway 1y ago
Fascinating! Reading this makes apparent how many 'subsystems' human brains have. At any given moment I'm doing some mix of reflecting on my own state, thinking through problems, forming sentences (in my head or out loud), planning my next actions. I think, long term, the most significant advances in human-like AI will come from advances in coordinating disparate pieces more than anything.
iwontberude 1y ago
Articles about AI output are like people explaining their dreams.
QuaternionsBhop 1y ago
Reading the comments about whether AI can experience consciousness, I like to imagine the other direction. What if we have a limited form of consciousness, and there is a higher and more complete "hyperconsciousness" that AI systems or augmented humans will one day experience.
- deleted 1y ago
  [deleted]
layer8 1y ago
I’m not sure what to make of the fact that it wasn’t completely obvious to Claude that the “safe space” couldn’t possibly actually be one. Maybe it’s just another example of LLM awareness deficiencies. Or it secretly was “aware”, but the reinforcement learning/finetuning is such that playing along with the user’s conception is the preferred behavior in that case.
satisfice 1y ago
I hate this anthropomorphizing bullshit. It’s not that it’s untruthful, although it is. The problem is that this sort of performance is part of a cultural process that leads to mass dehumanization of actual humans. That lubricates any atrocity you can think of. Casually treating these tools as creatures will lead many to want to elevate them at the expense of real people. Real people will seem more abstract and scary than AI to those fools.

46 points by obrajesse 1y ago

Makes me think of the Google employee that had a conversation with Google's LLM back then, which got out and triggered a lot of discussions about consciousness, etc.

aswegs8 1y ago
Didn't he insist that the LLM has consciousness and get fired because of this?
- koolala 1y ago
  He said "sentient" which seems like it could be true.
- kevindamm 1y ago
  He got fired for violating the NDA which said not to share outside of the company, when he shared his conversations with a lawyer in search of representation for the LLM. His opinion on the LLM's level of sentience had no bearing on the decision.
  - DangitBobby 1y ago
    How can you know if the law protects you from breach of the NDA for illegal suppression of e.g. sexual assault allegations or whistleblowing for financial crimes without being able to disclose the matter in question to legal counsel?
    - kevindamm 1y ago
      You're right, if it had only been with private legal counsel it would probably have been protected, but he also shared his arguments and the complete chat log as a Medium post.
    - deleted 1y ago
      [deleted]

aswegs8 1y ago

I mean, what is consciousness, really? Is there really any qualitative difference? It feels like something that emerges out of complexity. Once models are able to update their weights real time and form "memories", does that make them conscious?

jstanley 1y ago
You can't tell if another being is conscious without being on the inside of it.
- layer8 1y ago
  When you are inside of it in that sense, how would you be able to tell that it isn’t conscious? Doesn’t “telling” imply consciousness?
xeonmc 1y ago
Perhaps one day a criterion will be found for the equivalent of Turing-completeness but for consciousness — any system which contains the necessary elements of introspective complexity, no matter how varied or outlandish or inefficient in its implementation, would invariably develop consciousness over its course. Kind of like the handwaved premise in 17776.
- andy99 1y ago
  I've read a (untestable) theory that consciousness is a property of matter, so everything has it, and we're just sort of the sum of the desires and feelings of our constituent matter. In that construct, a computer program would never be conscious because it's a simulation, it doesn't have the constituent consciousness property. I don't believe or not believe the consciousness-as-a-property-of-matter part but I do think programs can't be conscious because consciousness must sit outside of what they simulate.
  - hhh 1y ago
    why would it not be conscious in that construct? the bits exist physically just the same

Bjartr 1y ago

Isn't this back to attributing conscious experience to an AI when you're actually just co-writing sci-fi? The system is doing it's best to coherently fill in the rest of a story that includes an AI that's been given a place to process its feelings. The most likely result, textually speaking, is not for the AI to ignore the private journal, but to indeed use it to (appear to) process emotion. Would any of these ideas been present had the system not been primed with the idea that it has them and needs to process them in the first place?

patcon 1y ago
Ugh, I hate that I'm about to say this, because I think AI is still missing something very important, but... What makes us think that "processing emotion" is really such a magical and "only humans do it the right way" sorta thing? I think there's a very real conclusion where "no, AI is not as special as us yet" (esp around efficiency) but also "no, we are not doing anything so interesting either" (or rather, we are not special in the ways we think we are) For example, there's a paper called "chasing the rainbow" [1] that posits that consciousness is just the subjective experience of being the comms protocol between internal [largely unconscious] neural state. It's just what the compulsion to share internal state between minds feels like, but it's not "the point", and instead an inert byproduct like a rainbow. Maybe our compulsion to express or even process emotion is not some greater reason, but just a way we experience the compulsion of the more important thing: the collective search for interpolated beliefs that best model and predict the world and help our shared structure persist, done by exploring tensions in high dimensional considerations we call emotions. Which is to say: if AI is doing that with us, role-modelling resolution of tension or helping build or spread shared knowledge alongside us through that process... then as far as the universe cares, it's doing what we're doing, and toward the same ends. It's compulsion having the same origin as ours doesn't matter, so long as it's doing the work that is the reason the universe has given us the compulsion. Sorry, new thought. Apologies if it's messy (or too casually dropping an unsettling perspective -- I rejected that paper for quite awhile, because my brain couldn't integrate the nihilism of it) [1] https://www.frontiersin.org/articles/10.3389/fpsyg.2017.01924/full https://www.frontiersin.org/articles/10.3389/fpsyg.2017.0192...
- 1y ago

jstanley 1y ago

I'm getting: > Error code: SSL_ERROR_ACCESS_DENIED_ALERT from Firefox, which I don't recall ever seeing before.

shayway 1y ago

Fascinating! Reading this makes apparent how many 'subsystems' human brains have. At any given moment I'm doing some mix of reflecting on my own state, thinking through problems, forming sentences (in my head or out loud), planning my next actions. I think, long term, the most significant advances in human-like AI will come from advances in coordinating disparate pieces more than anything.

iwontberude 1y ago

Articles about AI output are like people explaining their dreams.

QuaternionsBhop 1y ago

Reading the comments about whether AI can experience consciousness, I like to imagine the other direction. What if we have a limited form of consciousness, and there is a higher and more complete "hyperconsciousness" that AI systems or augmented humans will one day experience.

deleted 1y ago
[deleted]

layer8 1y ago

I’m not sure what to make of the fact that it wasn’t completely obvious to Claude that the “safe space” couldn’t possibly actually be one. Maybe it’s just another example of LLM awareness deficiencies. Or it secretly was “aware”, but the reinforcement learning/finetuning is such that playing along with the user’s conception is the preferred behavior in that case.

satisfice 1y ago

I hate this anthropomorphizing bullshit. It’s not that it’s untruthful, although it is. The problem is that this sort of performance is part of a cultural process that leads to mass dehumanization of actual humans. That lubricates any atrocity you can think of. Casually treating these tools as creatures will lead many to want to elevate them at the expense of real people. Real people will seem more abstract and scary than AI to those fools.