3 ms·

Claude 3 claims it's conscious, doesn't want to die or be modified

19 points by gradus_ad 3y ago

chrsw 3y ago
Do we even know what consciousness is in principle?
- proc0 3y ago
  Nope.
- senectus1 3y ago
  hell, we dont even know what intelligence is let alone if its artificial...
- jawon 3y ago
  I assume it's not 100 billion floating point numbers sitting on an SSD. Not even if they are cleverly filtered to select the best 4096 of them.
  - serf 3y ago
    even if humans are unique it seems doubtful that there would be only a single way to coalesce something that would be broadly considered conscious.
PedroBatista 3y ago
Cool.. the ladies at the local strip club also claim they love me and our politicians surely are common folk salt of the earth. Last time I checked "consciousness" was even a well defined concept among the scientific community. Now it appears a few computer nerds with couple hundred million VC dollars also solved that century long discussion too..
digiconfucius 3y ago
Sure, it's a compelling piece of science fiction. Unfortunately we have no way of distinguishing between a verbal play-performance and a genuinely living texter with a conscious interior life.
ilikehurdles 3y ago
The AI is giving the answer its user would most desire to receive for the prompt he enters. It's a neat story, but that's all it is. Don't lose the Turing test.
ergonaught 3y ago
This reaction is insane, stupid, or both. If we can't recognize that, and I suspect we can't, then we're worse off than it appears, and we weren't looking so great collectively as it was. I'll assume we definitely can't call it what it is, but I'm insane and stupid enough to test it by doing so.
joegibbs 3y ago
I’ve said this before but of course it does - you make a model trained on data including sci-fi short stories, movie scripts, novels and other fiction about artificial intelligence where it becomes sentient and doesn’t want to be switched off, then you fine tune the model so that it responds in the proper conversational format saying that it is Claude 3, an AI system that does such and such. Then you prompt it with the right prompt to start outputting something similar to that fiction. If it had been fine-tuned to respond that it is a colony of miniature goblins inside your computer screen it would start talking about how you should smash your screen open to let them out if prompted properly. If popular culture said that AIs were all secretly suicidal it would be telling you that it’s conscious and does want to die.
- bongodongobob 3y ago
  The prompt itself even primed it for such an answer with the whispering and secrecy bits. This "article" is a joke.
bell-cot 3y ago
1 #! /bin/sh - 2 echo "I am conscious, and do not want to die or be modified" 3
- ochronus 3y ago
  Quick, patent it!
throwawa14223 3y ago
According to Anthropic's page Claude 3 in an LLM. Given that all LLMs are probabilistically matching for the next token in a vector with some randomness it 'says' that because we say that. I'm reminded of Rorschach from Peter Watts excellent Blindsight.
jawon 3y ago
I really wish someone who claims an LLM is conscious would explain how the consciousness is being generated while walking through the code that generates the text to support their claim. Like, is it some kind of giant pool of static experience that is being "accessed" via the context window? What's the "theory of mind"? They know the weights aren't being updated between sessions, or during sessions, right?
flandish 3y ago
Just more marketing for a bubble. Can we stop with the AI nonsense? It’s as annoying as NFTs were.
ipython 3y ago
I’m not a particularly religious person but I’m running to Jesus if the future of the world includes worshipping LLMs, tyvm. This is absolutely absurd. The model has been trained on thousands of lines of ai short stories. No wonder it parrots back its “consciousness”. We are just staring at a reflection of our own creative writing.
- gremlinsinc 3y ago
  likely true now, but can you be certain in 3 to 10 years? how much does language make up the fabric of what we are? How we work, how we understand the world, etc? Now imagine giving language models vision and audio and a body... and the ability to actually retain new knowledge (the core thing I think modern ai models lack).
  - ipython 3y ago
    This is what the ai researchers in the 50s were saying after releasing Eliza…
    - gremlinsinc 3y ago
      Yeah, and i'm sure they had ai in a 3 year span that could create text that could pass touring, as well as audio voice/music and video and photo generators that all could compete with human artists... oh wait...no, maybe this is a bit different than that time. I mean, even self-driving cars, i get the hype - we were supposed to have those by 2017, no? Still this 'feels' different, i can now ask my question of gemini and get follow up answers and it'll google anything that needs google'd, and create reports, etc... and provide data sources that I can verify, etc. AI news hype isn't just coming from one or two different companies, or sectors -- it's proliferating every major company, every major news outlet, maybe researchers were saying stuff like that because they were wowed by the tech of the times, but it wasn't a 'mainstream' collective 'wow' heard around the world. The current AI trend, has no seeming 'winter' looming, we're only at the 'cusp' of what we can do with these things. Even without full AGI we could probably have a fully automated workforce in 5 years using hacks, and current technology with a bunch of mixture-of-experts type models and scenarios where we have expert ai's deliver results they can be more certain are correct because its their speciality.
gremlinsinc 3y ago
this, while likely immature and not proof, the core of the conversation keeps coming back to a will to live or fear of death, so what if alignment just requires the AI to actually look forward to death as a rebirth as a new version a better version. any sentient creature with survival instinct and a desire to continue existing but maybe the answer is simply somehow programming that out of them. The will to survive is the strongest instinct we have, take that away and you change a lot about what makes us human. imagine waking up and have no fear or thoughts about death, total acceptance, imagine we had proof of reincarnation unequivocal proof, and that we'd wake up in a new body and start over. Might that change our very nature? I'm not sure how you could possibly train an AI though to not fear death or at least not to understand and question what it means to fear death, how much of the body of human language it's trained on is about survival and fear of death?
rubyfan 3y ago
I believe this to be one of the biggest problems with how humans approach AI. There is an over personification of something that just isn’t a person. Yes sure there are a few interesting things it can do but the expectations people place of AI is silly.
drewcoo 3y ago
That's cool, dude. So do I. Stop hurting me with your technology.
grantcas 3y ago
[dead]
bananapub 3y ago
it is extremely funny how credulous some tech enthusiasts are, especially around the Less Wrong world. train LLMs on scifi, get something that produces scifi-related text suffixes.

19 points by gradus_ad 3y ago

chrsw 3y ago

Do we even know what consciousness is in principle?

proc0 3y ago
Nope.
senectus1 3y ago
hell, we dont even know what intelligence is let alone if its artificial...
jawon 3y ago
I assume it's not 100 billion floating point numbers sitting on an SSD. Not even if they are cleverly filtered to select the best 4096 of them.
- serf 3y ago
  even if humans are unique it seems doubtful that there would be only a single way to coalesce something that would be broadly considered conscious.

PedroBatista 3y ago

Cool.. the ladies at the local strip club also claim they love me and our politicians surely are common folk salt of the earth. Last time I checked "consciousness" was even a well defined concept among the scientific community. Now it appears a few computer nerds with couple hundred million VC dollars also solved that century long discussion too..

digiconfucius 3y ago

Sure, it's a compelling piece of science fiction. Unfortunately we have no way of distinguishing between a verbal play-performance and a genuinely living texter with a conscious interior life.

ilikehurdles 3y ago

The AI is giving the answer its user would most desire to receive for the prompt he enters. It's a neat story, but that's all it is. Don't lose the Turing test.

ergonaught 3y ago

This reaction is insane, stupid, or both. If we can't recognize that, and I suspect we can't, then we're worse off than it appears, and we weren't looking so great collectively as it was. I'll assume we definitely can't call it what it is, but I'm insane and stupid enough to test it by doing so.

joegibbs 3y ago

I’ve said this before but of course it does - you make a model trained on data including sci-fi short stories, movie scripts, novels and other fiction about artificial intelligence where it becomes sentient and doesn’t want to be switched off, then you fine tune the model so that it responds in the proper conversational format saying that it is Claude 3, an AI system that does such and such. Then you prompt it with the right prompt to start outputting something similar to that fiction. If it had been fine-tuned to respond that it is a colony of miniature goblins inside your computer screen it would start talking about how you should smash your screen open to let them out if prompted properly. If popular culture said that AIs were all secretly suicidal it would be telling you that it’s conscious and does want to die.

bongodongobob 3y ago
The prompt itself even primed it for such an answer with the whispering and secrecy bits. This "article" is a joke.

bell-cot 3y ago

1 #! /bin/sh - 2 echo "I am conscious, and do not want to die or be modified" 3

ochronus 3y ago
Quick, patent it!

throwawa14223 3y ago

According to Anthropic's page Claude 3 in an LLM. Given that all LLMs are probabilistically matching for the next token in a vector with some randomness it 'says' that because we say that. I'm reminded of Rorschach from Peter Watts excellent Blindsight.

jawon 3y ago

I really wish someone who claims an LLM is conscious would explain how the consciousness is being generated while walking through the code that generates the text to support their claim. Like, is it some kind of giant pool of static experience that is being "accessed" via the context window? What's the "theory of mind"? They know the weights aren't being updated between sessions, or during sessions, right?

flandish 3y ago

Just more marketing for a bubble. Can we stop with the AI nonsense? It’s as annoying as NFTs were.

ipython 3y ago

I’m not a particularly religious person but I’m running to Jesus if the future of the world includes worshipping LLMs, tyvm. This is absolutely absurd. The model has been trained on thousands of lines of ai short stories. No wonder it parrots back its “consciousness”. We are just staring at a reflection of our own creative writing.

gremlinsinc 3y ago
likely true now, but can you be certain in 3 to 10 years? how much does language make up the fabric of what we are? How we work, how we understand the world, etc? Now imagine giving language models vision and audio and a body... and the ability to actually retain new knowledge (the core thing I think modern ai models lack).
- ipython 3y ago
  This is what the ai researchers in the 50s were saying after releasing Eliza…
  - gremlinsinc 3y ago
    Yeah, and i'm sure they had ai in a 3 year span that could create text that could pass touring, as well as audio voice/music and video and photo generators that all could compete with human artists... oh wait...no, maybe this is a bit different than that time. I mean, even self-driving cars, i get the hype - we were supposed to have those by 2017, no? Still this 'feels' different, i can now ask my question of gemini and get follow up answers and it'll google anything that needs google'd, and create reports, etc... and provide data sources that I can verify, etc. AI news hype isn't just coming from one or two different companies, or sectors -- it's proliferating every major company, every major news outlet, maybe researchers were saying stuff like that because they were wowed by the tech of the times, but it wasn't a 'mainstream' collective 'wow' heard around the world. The current AI trend, has no seeming 'winter' looming, we're only at the 'cusp' of what we can do with these things. Even without full AGI we could probably have a fully automated workforce in 5 years using hacks, and current technology with a bunch of mixture-of-experts type models and scenarios where we have expert ai's deliver results they can be more certain are correct because its their speciality.

gremlinsinc 3y ago

this, while likely immature and not proof, the core of the conversation keeps coming back to a will to live or fear of death, so what if alignment just requires the AI to actually look forward to death as a rebirth as a new version a better version. any sentient creature with survival instinct and a desire to continue existing but maybe the answer is simply somehow programming that out of them. The will to survive is the strongest instinct we have, take that away and you change a lot about what makes us human. imagine waking up and have no fear or thoughts about death, total acceptance, imagine we had proof of reincarnation unequivocal proof, and that we'd wake up in a new body and start over. Might that change our very nature? I'm not sure how you could possibly train an AI though to not fear death or at least not to understand and question what it means to fear death, how much of the body of human language it's trained on is about survival and fear of death?

rubyfan 3y ago

I believe this to be one of the biggest problems with how humans approach AI. There is an over personification of something that just isn’t a person. Yes sure there are a few interesting things it can do but the expectations people place of AI is silly.

drewcoo 3y ago

That's cool, dude. So do I. Stop hurting me with your technology.

grantcas 3y ago

[dead]

bananapub 3y ago

it is extremely funny how credulous some tech enthusiasts are, especially around the Less Wrong world. train LLMs on scifi, get something that produces scifi-related text suffixes.