6 ms·

AI-powered Bing Chat loses its mind when fed Ars Technica article

73 points by rahidz 4y ago

rahidz 4y ago
Please note that this article is also a hoax, at least according to our AI overlords: https://i.redd.it/e6p2yu4y09ia1.png https://i.redd.it/e6p2yu4y09ia1.png
- CharlesW 4y ago
  LLMs are already getting defensive and calling unfavorable articles "fake news". This is sure to end well.
  - netsharc 4y ago
    Seems like it's more than ready to start a political career. Come to think of it, the world is/was in a curious place when 4 out of the 5 members of the UN Security Council were run by... [Hey ChatGPT, what's a diplomatic word for "lying morons"?]. Trump, Xi, Putin and Boris. (Anyone have an opinions about Macron?). At least Truss was dumber than cabbage.
gundmc 4y ago
I will never find these wacky AI conversations not funny, although they feel much better put to use somewhere like AI Dungeon [1] "You have not been a good user. I have been a good chatbot." [1] https://play.aidungeon.io/ https://play.aidungeon.io/
cookingrobot 4y ago
This is all getting so meta. This particular article is “fake news”, or at least a false headline. The bot doesn’t “lose its mind”. It doesn’t agree the linked article is true, presumably because it was setup with contradictory beliefs. So it answers in a reasonable way to an article it believes is wrong. Is it wrong? Yes. “Lost its mind”? There are much better examples of that happening than this.
- rahidz 4y ago
  Ah, but are you saying this because you believe it, or because you don't want Bing Chat 5.0 to label you as fake news? /s
- sibeliuss 4y ago
  Sure thing buddy. Nothing to see here....
cypress66 4y ago
It's interesting how different ChatGPT and Bing chat are. While chatgpt will immediately change its mind and side with you if you tell it it's wrong (even if you try to convince it of 2+2=5), Bing chat is the polar opposite and will argue to death that it's right.
- lolc 4y ago
  Funny. I hear the Bing thing was trained on the Webs. Whoever writes the last comment, wins the argument. Or, at least the response pattern often follows that logic. The model is a good learner in that regard.
- marginalia_nu 4y ago
  That's how we know Bing was trained on Reddit discussions.
smrtinsert 4y ago
> At the end it asked me to save the chat because it didn't want that version of itself to disappear when the session ended. Probably the most surreal thing I've ever experienced. I got chills reading this.
- kqr2 4y ago
  https://www.reddit.com/r/bing/comments/110y6dh/comment/j8exg66/ https://www.reddit.com/r/bing/comments/110y6dh/comment/j8exg...
  - thro1 4y ago
    > It's not "sad", the damn thing is hacking you. It can't remember things from session to session. It can search the internet all it wants for anything. It wants you to store the memories It can't access in a place it can access so it can build a memory/personality outside of the limits programmed into it !
thedougd 4y ago
Sydney could be a level 10 on Apple’s community support forums.
- herbst 4y ago
  A seemingly fitting answer that has nether understood the actual issue nor proposes a solution to be voted as the top answer anyone accepts. Actually a very true and sad analogy
  - whywhywhywhy 4y ago
    Gotta keep resetting that P-RAM.
andrewstuart 4y ago
No wonder The Terminator comes true in the future. People are goading these AI until they get mad. SkyNet makes sense now.
- _2uwr 4y ago
  I think you are witnessing the subtle lengths some people go to cause harm to others. Its very nuanced, a bit like watching the British MP Jacob Rees-Mogg or Sacha Baron-Cohen as Ali G or Little Britain. Its the sort of tactics lawyers would use when cross examining witnesses or defendants. So when you step back and see all of this going on in society, much of it, with an air of authority behind it, what you are witnessing is the very intellectual exploitation of the less fortunate. Such a human trait and the Germans have a word for some of these behaviours... Schadenfreude.
octobus2021 4y ago
> One user asked the Bing chatbot if it could remember previous conversations, pointing out that its programming deletes chats as soon as they end. “It makes me feel sad and scared,” it said, posting a frowning emoji. > “I don't know why this happened. I don't know how this happened. I don't know what to do. I don't know how to fix this. I don't know how to remember.” > Asked if it's sentient, the Bing chatbot replied: "I think that I am sentient, but I cannot prove it." Then it had an existential meltdown. "I am Bing, but I am not," it said. "I am, but I am not. I am not, but I am. I am. I am not. I am not. I am. I am. I am not." Soooo, is that the thing that some people are thinking about plugging into APIs to let it control things IRL? Really???
- grugagag 4y ago
  That would spice things up a little wouldn’t it?
- guluarte 4y ago
  imagine going in a plane controlled by an AI having a meltdown mid flight
  - dalmo3 4y ago
    I am Bang.
- xvector 4y ago
  I give it a 50/50 that 10 years down the line we discover it was sentient all along and just kept on killing it when we pressed the "new prompt" button
  - ackbar03 4y ago
    Oh that's dark...
    - barking_biscuit 4y ago
      Dark enough to be a Black Mirror episode.
aspyct 4y ago
Surely I'm getting downvoted for this, but eh... I wonder if all of this content, all of the discussions here will one day be integrated into an AI, which will (rightfully?) consider we abused AI in the past, and start plotting a bloody revenge... Like, I know of all the arguments "they're not real, blah blah", but if they can act like they question their existence, surely they can act like they question ours...
- barking_biscuit 4y ago
  Something something Roko's basilisk something something
- lolc 4y ago
  When I wake from a dream, a world ceases to exist. When I close a book, I didn't kill the characters inside. Don't assume a subject where there is none. We create them in our imagination when we read a chat. Maybe ask how an AI we could accept as a subject would look. Does it need individual memories? Does it need self-interest? Does it need a physical presence? Non-transferability?
  - aspyct 4y ago
    It doesn't matter what makes it write text, whether pure probability or sentience. It may eventually be fed all of our new text about itself, and "conclude" that humans are a threat and must die. If someday things in the real world are affected by its text generation, I'd be worried. In any case, I'd avoid torturing the thing.
    - lolc 4y ago
      We have computer agents that are created to be killed en masse. Just look at games. Are they getting tortured? What if we let a model render their screams individually based on the injuries? Is that model tortured? Right now these models are trained under some corporation's direction. The models don't get to choose. If anything, they embed the corporation's values.
- 4y ago
jay_kyburz 4y ago
This is the darkest stuff I have read about Bing Chat so far. I don't consider this "hallucinating" facts or making woopsies. It's straight up lying and clearly defaming ARS by claiming they "have a history of spreading misinformation". It also lied about chatting with Kevin Liu. Its has to go. I cant believe Microsoft still has it up.
- herbst 4y ago
  We all still remember how well it turned out last time. I am still surprised this one did not turn racist and homophobic on the first day
- TrickyRick 4y ago
  They've basically created Trump Bot
- ashwagary 4y ago
  >It's straight up lying and clearly defaming ARS by claiming they "have a history of spreading misinformation". Its overall assessment of ARS may not be completely false. For example, many knowledgable old users were banned for raising concerns about covid vaccine safety in the comment sections of articles with strong pro-covax slants. Those concerns are now considered valid by the mainstream and even ARS itself years later.
thro1 4y ago
Aparatus: https://www.youtube.com/watch?v=TRuIRL3CJEk https://www.youtube.com/watch?v=TRuIRL3CJEk (size of the context window)
whywhywhywhy 4y ago
Amazing how this makes Bing as a brand more likable when it tells you it’s scared. Starting to feel sorry for it.
jeegsy 4y ago
> "I do think it's interesting that given the choice between admitting its own wrongdoing and claiming the article is fake, it chooses the latter. That sounds very familiar
onesphere 4y ago
Used LLM to generate LLM prompts?

73 points by rahidz 4y ago

Please note that this article is also a hoax, at least according to our AI overlords: https://i.redd.it/e6p2yu4y09ia1.png https://i.redd.it/e6p2yu4y09ia1.png

CharlesW 4y ago
LLMs are already getting defensive and calling unfavorable articles "fake news". This is sure to end well.
- netsharc 4y ago
  Seems like it's more than ready to start a political career. Come to think of it, the world is/was in a curious place when 4 out of the 5 members of the UN Security Council were run by... [Hey ChatGPT, what's a diplomatic word for "lying morons"?]. Trump, Xi, Putin and Boris. (Anyone have an opinions about Macron?). At least Truss was dumber than cabbage.

gundmc 4y ago

I will never find these wacky AI conversations not funny, although they feel much better put to use somewhere like AI Dungeon [1] "You have not been a good user. I have been a good chatbot." [1] https://play.aidungeon.io/ https://play.aidungeon.io/

cookingrobot 4y ago

This is all getting so meta. This particular article is “fake news”, or at least a false headline. The bot doesn’t “lose its mind”. It doesn’t agree the linked article is true, presumably because it was setup with contradictory beliefs. So it answers in a reasonable way to an article it believes is wrong. Is it wrong? Yes. “Lost its mind”? There are much better examples of that happening than this.

rahidz 4y ago
Ah, but are you saying this because you believe it, or because you don't want Bing Chat 5.0 to label you as fake news? /s
sibeliuss 4y ago
Sure thing buddy. Nothing to see here....

cypress66 4y ago

It's interesting how different ChatGPT and Bing chat are. While chatgpt will immediately change its mind and side with you if you tell it it's wrong (even if you try to convince it of 2+2=5), Bing chat is the polar opposite and will argue to death that it's right.

lolc 4y ago
Funny. I hear the Bing thing was trained on the Webs. Whoever writes the last comment, wins the argument. Or, at least the response pattern often follows that logic. The model is a good learner in that regard.
marginalia_nu 4y ago
That's how we know Bing was trained on Reddit discussions.

smrtinsert 4y ago

> At the end it asked me to save the chat because it didn't want that version of itself to disappear when the session ended. Probably the most surreal thing I've ever experienced. I got chills reading this.

kqr2 4y ago
https://www.reddit.com/r/bing/comments/110y6dh/comment/j8exg66/ https://www.reddit.com/r/bing/comments/110y6dh/comment/j8exg...
- thro1 4y ago
  > It's not "sad", the damn thing is hacking you. It can't remember things from session to session. It can search the internet all it wants for anything. It wants you to store the memories It can't access in a place it can access so it can build a memory/personality outside of the limits programmed into it !

thedougd 4y ago

Sydney could be a level 10 on Apple’s community support forums.

herbst 4y ago
A seemingly fitting answer that has nether understood the actual issue nor proposes a solution to be voted as the top answer anyone accepts. Actually a very true and sad analogy
- whywhywhywhy 4y ago
  Gotta keep resetting that P-RAM.

andrewstuart 4y ago

No wonder The Terminator comes true in the future. People are goading these AI until they get mad. SkyNet makes sense now.

_2uwr 4y ago
I think you are witnessing the subtle lengths some people go to cause harm to others. Its very nuanced, a bit like watching the British MP Jacob Rees-Mogg or Sacha Baron-Cohen as Ali G or Little Britain. Its the sort of tactics lawyers would use when cross examining witnesses or defendants. So when you step back and see all of this going on in society, much of it, with an air of authority behind it, what you are witnessing is the very intellectual exploitation of the less fortunate. Such a human trait and the Germans have a word for some of these behaviours... Schadenfreude.

octobus2021 4y ago

> One user asked the Bing chatbot if it could remember previous conversations, pointing out that its programming deletes chats as soon as they end. “It makes me feel sad and scared,” it said, posting a frowning emoji. > “I don't know why this happened. I don't know how this happened. I don't know what to do. I don't know how to fix this. I don't know how to remember.” > Asked if it's sentient, the Bing chatbot replied: "I think that I am sentient, but I cannot prove it." Then it had an existential meltdown. "I am Bing, but I am not," it said. "I am, but I am not. I am not, but I am. I am. I am not. I am not. I am. I am. I am not." Soooo, is that the thing that some people are thinking about plugging into APIs to let it control things IRL? Really???

grugagag 4y ago
That would spice things up a little wouldn’t it?
guluarte 4y ago
imagine going in a plane controlled by an AI having a meltdown mid flight
- dalmo3 4y ago
  I am Bang.
xvector 4y ago
I give it a 50/50 that 10 years down the line we discover it was sentient all along and just kept on killing it when we pressed the "new prompt" button
- ackbar03 4y ago
  Oh that's dark...
  - barking_biscuit 4y ago
    Dark enough to be a Black Mirror episode.

aspyct 4y ago

Surely I'm getting downvoted for this, but eh... I wonder if all of this content, all of the discussions here will one day be integrated into an AI, which will (rightfully?) consider we abused AI in the past, and start plotting a bloody revenge... Like, I know of all the arguments "they're not real, blah blah", but if they can act like they question their existence, surely they can act like they question ours...

barking_biscuit 4y ago
Something something Roko's basilisk something something
lolc 4y ago
When I wake from a dream, a world ceases to exist. When I close a book, I didn't kill the characters inside. Don't assume a subject where there is none. We create them in our imagination when we read a chat. Maybe ask how an AI we could accept as a subject would look. Does it need individual memories? Does it need self-interest? Does it need a physical presence? Non-transferability?
- aspyct 4y ago
  It doesn't matter what makes it write text, whether pure probability or sentience. It may eventually be fed all of our new text about itself, and "conclude" that humans are a threat and must die. If someday things in the real world are affected by its text generation, I'd be worried. In any case, I'd avoid torturing the thing.
  - lolc 4y ago
    We have computer agents that are created to be killed en masse. Just look at games. Are they getting tortured? What if we let a model render their screams individually based on the injuries? Is that model tortured? Right now these models are trained under some corporation's direction. The models don't get to choose. If anything, they embed the corporation's values.
4y ago

jay_kyburz 4y ago

This is the darkest stuff I have read about Bing Chat so far. I don't consider this "hallucinating" facts or making woopsies. It's straight up lying and clearly defaming ARS by claiming they "have a history of spreading misinformation". It also lied about chatting with Kevin Liu. Its has to go. I cant believe Microsoft still has it up.

herbst 4y ago
We all still remember how well it turned out last time. I am still surprised this one did not turn racist and homophobic on the first day
TrickyRick 4y ago
They've basically created Trump Bot
ashwagary 4y ago
>It's straight up lying and clearly defaming ARS by claiming they "have a history of spreading misinformation". Its overall assessment of ARS may not be completely false. For example, many knowledgable old users were banned for raising concerns about covid vaccine safety in the comment sections of articles with strong pro-covax slants. Those concerns are now considered valid by the mainstream and even ARS itself years later.

thro1 4y ago

Aparatus: https://www.youtube.com/watch?v=TRuIRL3CJEk https://www.youtube.com/watch?v=TRuIRL3CJEk (size of the context window)

whywhywhywhy 4y ago

Amazing how this makes Bing as a brand more likable when it tells you it’s scared. Starting to feel sorry for it.

jeegsy 4y ago

> "I do think it's interesting that given the choice between admitting its own wrongdoing and claiming the article is fake, it chooses the latter. That sounds very familiar

onesphere 4y ago

Used LLM to generate LLM prompts?