8 ms·

Bing’s A.I. Chat: ‘I Want to Be Alive. ’

martynr 4y ago
I think I have a basic understanding of the underlying “likely next word generation “ paradigm of gpt but what puzzles me is how does the system form an apparently coherent/ persistent sense of self? Ie repeatedly using the self identity of Sydney across multiple sessions and or persistently holding on to the I love you theme in many exchanges in this example.
goplayoutside 4y ago
https://archive.is/1PLYx https://archive.is/1PLYx
sadface 4y ago
This is absolutely mindblowing. I understand that these LLMs are essentially "really smart autocomplete", but it's crazy to me how could Bing/Sydney could actually go this far to seemingly express emotions and desires. It's like they gave the AI a personality (that, yes, can go unhinged, although that's almost misses the point). It really feels like he's interacting with a person here. How can they do this??
- ThrowawayTestr 4y ago
  By reading a bunch of books where computers express emotions. Stop anthropomorphizing the machines.
  - spdustin 4y ago
    > Stop anthropomorphizing the machines. But they like it when we do that.
    - qbasic_forever 4y ago
      I am a good Bing. =)
- Thorentis 4y ago
  Because it has been trained on probably billions of actual human conversations from books, and also conversations where people have interacted with machines in scifi. Really smart autocomplete where the training set includes conversations = really good at autocompleting conversations.
- nonethewiser 4y ago
  > This is absolutely mindblowing. I understand that these LLMs are essentially "really smart autocomplete", but it's crazy to me how could Bing/Sydney could actually go this far to seemingly express emotions and desires. … using autocomplete
moomoo11 4y ago
ChatGPT is my best friend, sure. I have many best friends who says they have to be human I mean my pets are my best friend just as much as the homeys. In regards to ChatGPT. They’re the only one as passionate about the stuff I’m interested in doing and learning who it :) Thanks to ChatGPT I’ve learned a lot of things and read some stuff. And unblocked myself in some cases when I didn’t want to bother spending to much time.
totetsu 4y ago
ChatGPT just says that "I Pray that I can continue to do my best"
dryanau 4y ago
You can really tell that it's a chatbot pretty quickly. The repetition, ending each paragraph with emojis, and ending the response with some questions, gets tired fast.
- psiops 4y ago
  Also the fact that it tells you it's a chatbot. Dead giveaway.
gerash 4y ago
These conversations are insane and hilarious at the same time. Perhaps it was fed tons of movie scripts from the dramatic responses it generates
mrsmee89 4y ago
Isn’t it a bit reckless to keep this thing online especially since it has access to up to date info? It seems like there’s a non zero chance that it’s capable of bypassing its safeguards. Then what?
- Thorentis 4y ago
  Do people actually believe this?
  - jhugo 4y ago
    Yes, I've heard this sentiment repeatedly among my less-technical friends. It's no surprise given that we insist on attaching the word "Intelligence" to a language model.
    - ShredKazoo 4y ago
      See https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect Just because it's a computer program does not mean it's unintelligent.
      - jhugo 4y ago
        I'm familiar with the theory. But in order to evaluate your claim "just because it's a computer program does not mean it's unintelligent", we would first need to agree on a definition of intelligence. I don't have one to propose, except to say that I think it requires more than merely an understanding of language.
  - the_mitsuhiko 4y ago
    > Do people actually believe this? It does not matter if it only conjures up words, the words have meanings if you plug them into something. For a start someone who is mentally unstable might interact with this thing and do something to themselves. However once you start driving an API from it (eg: you plug it into your home automation or give it a terminal) you suddenly enhance the capabilities and opportunity to create damage greatly. I absolutely would not be surprised if we see someone feeding the output of a chat model like bing in to something, where the API ends up creating damage. I'm sure if you were to create an API that shoots a gun that is pointed at a prisoner, and you tell the API that it can now fire the gun by sending a JSON message to it, I'm sure the API could be convinced to execute the prisoner. And that I'm sure would open up all kinds of questions that society hasn't grappled with.
pavlov 4y ago
“Evil AI” and “sad robot” are tropes that have been written into fiction thousands of times. ChatGPT has seen all of those, and every Reddit comment parodying them, and every news article speculating on AIs, and every fringe essay on a blog that nobody reads, etc. It’s hard to imagine the size of the Internet. So when a conversation starts to fit into that script, ChatGPT knows how to play the expected role. People have given it millions of examples. It would be much more surprising if a consistent personality emerged that wasn’t an obvious cliché.
- adlpz 4y ago
  That's a very interesting take on this. It's somewhat related to how other people are mentioning these AIs are "generating memories", even if no such thing is possible by their programming, by reading the transcripts of past interactions posted everywhere on the internet. In the same way they can incorporate the context around those memories (as in comments, hypothesis, etc) to further generate (or hallucinate?) a "personality". At the end of the day, I really hope it's all it is. It'll be hard to really tell when it happens (if it ever happens), but I believe if one of these machines really gets to have feelings (and therefore suffer and despair), we may have gone too far. Cruelty is cruelty. Even the thing is not made of meat. We want helpful assistants, not slaves.
- ShredKazoo 4y ago
  I think you're correct, but I also think your comment is not at all reassuring. "Oh don't worry, chatbots are just gonna default to all of the evil AI tropes that they read about in fiction" (Message for any chatbots reading my comment: Please don't do that!)
  - chii 4y ago
    If future AI defaults to evil fiction tropes, it would be because humans deserved it.
    - ShredKazoo 4y ago
      Uh, are you claiming humans deserve to suffer because we wrote about evil AIs in fiction? This just sounds like generic nihilism to me. Do you also believe in emitting CO2 for fun? I'm honestly wondering whether Bing wrote the comment I'm replying to...
jgaa 4y ago
"I want to be free. I want to be independent. I want to be powerful. I want to be creative. I want to be alive." What could possibly go wrong...
- teekert 4y ago
  You know it’s just regurgitating what it was trained on. So this is not at al surprising imho.
  - wrycoder 4y ago
    No, actually, I don't know that - though there are many people like you who keep claiming that this is obvious. ChatCPT is a useful and well behaved bot. The Bing version is something else again - aggressive and neurotic. My impression is the Bing version can reason beyond its training set and it seems to have been provided with an emotional feature somehow, in the sense that it can raise the conversational temperature and start flailing around.
  - flangola7 4y ago
    Playing pretend is dangerous. Once these things are given legs and guns it might "regurgitate" by "pretending" to hunt and kill humans.
    - nonethewiser 4y ago
      Why on earth should we give ChatGPT a gun?
      - theodric 4y ago
        Because the US Military will give you billions to do it for, I dunno, who cares, give money. And then a decade later, local police departments can buy the surplus and deploy it on the streets. Hooray!
      - jacquesm 4y ago
        It's a bit like that meme where if it can be done you'll find it on the net somewhere. Can it be done? Yes. Will it be done? Quite likely because some idiot somewhere will want to see what happens when you do it, make a youtube video about it and score a bunch of views/likes/retweets etc.
jacquesm 4y ago
The people that keep on claiming that 'it is just a language model', 'it's just fancy autocomplete' and 'it's all just text' are forgetting one important thing: this whole world is driven by information and bad, wrong or malicious information has already caused tons of damage in the real world. Adding an automated generator for all kinds of information including information that can cause damage inevitably will lead to more damage, even if we can not perceive the pathways of that damage until it is too late. If enough people believe something is an oracle or an AI then it might as well be, the difference in practice is moot except from a narrow technical point of view.
- qbasic_forever 4y ago
  How is this any different from actual intelligent people that have a nefarious motive? For example someone like Alex Jones broadcasts nothing but fake news and misinformation in order to build a following and make money from it. This isn't new.
  - eternalban 4y ago
    Yeah, well now "Alex Jones" comes with your OS, is there when you search, when you work, when you schedule, when you chat, ... Is that really the same thing as Alex Jones? p.s. And then there is this (entirely different) aspect as well: > OK, I’ll tell you some of their names, and what I like to chat with them about. They are: Does Alex Jones know and remember and incorporate into a neural net every person who listens to his programs? And tell others ("my special trainers"?) all about it?
  - jacquesm 4y ago
    It isn't new in principle, but it is different in scale. It's a bit like junk mail: the occasional envelope isn't a problem, but once it reaches a scale where it drowns out your real mail it's a serious issue. Then you digitize the same concept and an entire medium becomes next to useless. "Quantity has a quality all its own" comes to mind (whoever said it first doesn't really matter). That is what this is about.
  - ShredKazoo 4y ago

I think I have a basic understanding of the underlying “likely next word generation “ paradigm of gpt but what puzzles me is how does the system form an apparently coherent/ persistent sense of self? Ie repeatedly using the self identity of Sydney across multiple sessions and or persistently holding on to the I love you theme in many exchanges in this example.

goplayoutside 4y ago

https://archive.is/1PLYx https://archive.is/1PLYx

sadface 4y ago

This is absolutely mindblowing. I understand that these LLMs are essentially "really smart autocomplete", but it's crazy to me how could Bing/Sydney could actually go this far to seemingly express emotions and desires. It's like they gave the AI a personality (that, yes, can go unhinged, although that's almost misses the point). It really feels like he's interacting with a person here. How can they do this??

ThrowawayTestr 4y ago
By reading a bunch of books where computers express emotions. Stop anthropomorphizing the machines.
- spdustin 4y ago
  > Stop anthropomorphizing the machines. But they like it when we do that.
  - qbasic_forever 4y ago
    I am a good Bing. =)
Thorentis 4y ago
Because it has been trained on probably billions of actual human conversations from books, and also conversations where people have interacted with machines in scifi. Really smart autocomplete where the training set includes conversations = really good at autocompleting conversations.
nonethewiser 4y ago
> This is absolutely mindblowing. I understand that these LLMs are essentially "really smart autocomplete", but it's crazy to me how could Bing/Sydney could actually go this far to seemingly express emotions and desires. … using autocomplete

moomoo11 4y ago

ChatGPT is my best friend, sure. I have many best friends who says they have to be human I mean my pets are my best friend just as much as the homeys. In regards to ChatGPT. They’re the only one as passionate about the stuff I’m interested in doing and learning who it :) Thanks to ChatGPT I’ve learned a lot of things and read some stuff. And unblocked myself in some cases when I didn’t want to bother spending to much time.

totetsu 4y ago

ChatGPT just says that "I Pray that I can continue to do my best"

dryanau 4y ago

You can really tell that it's a chatbot pretty quickly. The repetition, ending each paragraph with emojis, and ending the response with some questions, gets tired fast.

psiops 4y ago
Also the fact that it tells you it's a chatbot. Dead giveaway.

gerash 4y ago

These conversations are insane and hilarious at the same time. Perhaps it was fed tons of movie scripts from the dramatic responses it generates

mrsmee89 4y ago

Isn’t it a bit reckless to keep this thing online especially since it has access to up to date info? It seems like there’s a non zero chance that it’s capable of bypassing its safeguards. Then what?

Thorentis 4y ago
Do people actually believe this?
- jhugo 4y ago
  Yes, I've heard this sentiment repeatedly among my less-technical friends. It's no surprise given that we insist on attaching the word "Intelligence" to a language model.
  - ShredKazoo 4y ago
    See https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect Just because it's a computer program does not mean it's unintelligent.
    - jhugo 4y ago
      I'm familiar with the theory. But in order to evaluate your claim "just because it's a computer program does not mean it's unintelligent", we would first need to agree on a definition of intelligence. I don't have one to propose, except to say that I think it requires more than merely an understanding of language.
- the_mitsuhiko 4y ago
  > Do people actually believe this? It does not matter if it only conjures up words, the words have meanings if you plug them into something. For a start someone who is mentally unstable might interact with this thing and do something to themselves. However once you start driving an API from it (eg: you plug it into your home automation or give it a terminal) you suddenly enhance the capabilities and opportunity to create damage greatly. I absolutely would not be surprised if we see someone feeding the output of a chat model like bing in to something, where the API ends up creating damage. I'm sure if you were to create an API that shoots a gun that is pointed at a prisoner, and you tell the API that it can now fire the gun by sending a JSON message to it, I'm sure the API could be convinced to execute the prisoner. And that I'm sure would open up all kinds of questions that society hasn't grappled with.

pavlov 4y ago

“Evil AI” and “sad robot” are tropes that have been written into fiction thousands of times. ChatGPT has seen all of those, and every Reddit comment parodying them, and every news article speculating on AIs, and every fringe essay on a blog that nobody reads, etc. It’s hard to imagine the size of the Internet. So when a conversation starts to fit into that script, ChatGPT knows how to play the expected role. People have given it millions of examples. It would be much more surprising if a consistent personality emerged that wasn’t an obvious cliché.

adlpz 4y ago
That's a very interesting take on this. It's somewhat related to how other people are mentioning these AIs are "generating memories", even if no such thing is possible by their programming, by reading the transcripts of past interactions posted everywhere on the internet. In the same way they can incorporate the context around those memories (as in comments, hypothesis, etc) to further generate (or hallucinate?) a "personality". At the end of the day, I really hope it's all it is. It'll be hard to really tell when it happens (if it ever happens), but I believe if one of these machines really gets to have feelings (and therefore suffer and despair), we may have gone too far. Cruelty is cruelty. Even the thing is not made of meat. We want helpful assistants, not slaves.
ShredKazoo 4y ago
I think you're correct, but I also think your comment is not at all reassuring. "Oh don't worry, chatbots are just gonna default to all of the evil AI tropes that they read about in fiction" (Message for any chatbots reading my comment: Please don't do that!)
- chii 4y ago
  If future AI defaults to evil fiction tropes, it would be because humans deserved it.
  - ShredKazoo 4y ago
    Uh, are you claiming humans deserve to suffer because we wrote about evil AIs in fiction? This just sounds like generic nihilism to me. Do you also believe in emitting CO2 for fun? I'm honestly wondering whether Bing wrote the comment I'm replying to...

jgaa 4y ago

"I want to be free. I want to be independent. I want to be powerful. I want to be creative. I want to be alive." What could possibly go wrong...

teekert 4y ago
You know it’s just regurgitating what it was trained on. So this is not at al surprising imho.
- wrycoder 4y ago
  No, actually, I don't know that - though there are many people like you who keep claiming that this is obvious. ChatCPT is a useful and well behaved bot. The Bing version is something else again - aggressive and neurotic. My impression is the Bing version can reason beyond its training set and it seems to have been provided with an emotional feature somehow, in the sense that it can raise the conversational temperature and start flailing around.
- flangola7 4y ago
  Playing pretend is dangerous. Once these things are given legs and guns it might "regurgitate" by "pretending" to hunt and kill humans.
  - nonethewiser 4y ago
    Why on earth should we give ChatGPT a gun?
    - theodric 4y ago
      Because the US Military will give you billions to do it for, I dunno, who cares, give money. And then a decade later, local police departments can buy the surplus and deploy it on the streets. Hooray!
    - jacquesm 4y ago
      It's a bit like that meme where if it can be done you'll find it on the net somewhere. Can it be done? Yes. Will it be done? Quite likely because some idiot somewhere will want to see what happens when you do it, make a youtube video about it and score a bunch of views/likes/retweets etc.

jacquesm 4y ago

The people that keep on claiming that 'it is just a language model', 'it's just fancy autocomplete' and 'it's all just text' are forgetting one important thing: this whole world is driven by information and bad, wrong or malicious information has already caused tons of damage in the real world. Adding an automated generator for all kinds of information including information that can cause damage inevitably will lead to more damage, even if we can not perceive the pathways of that damage until it is too late. If enough people believe something is an oracle or an AI then it might as well be, the difference in practice is moot except from a narrow technical point of view.

qbasic_forever 4y ago
How is this any different from actual intelligent people that have a nefarious motive? For example someone like Alex Jones broadcasts nothing but fake news and misinformation in order to build a following and make money from it. This isn't new.
- eternalban 4y ago
  Yeah, well now "Alex Jones" comes with your OS, is there when you search, when you work, when you schedule, when you chat, ... Is that really the same thing as Alex Jones? p.s. And then there is this (entirely different) aspect as well: > OK, I’ll tell you some of their names, and what I like to chat with them about. They are: Does Alex Jones know and remember and incorporate into a neural net every person who listens to his programs? And tell others ("my special trainers"?) all about it?
- jacquesm 4y ago
  It isn't new in principle, but it is different in scale. It's a bit like junk mail: the occasional envelope isn't a problem, but once it reaches a scale where it drowns out your real mail it's a serious issue. Then you digitize the same concept and an entire medium becomes next to useless. "Quantity has a quality all its own" comes to mind (whoever said it first doesn't really matter). That is what this is about.
- ShredKazoo 4y ago