4 ms·

I’m willing to be wrong, but, I don’t believe it. The user’s inputs are so weird, and the response is so out of left field… I would put money on this being fak

I’m willing to be wrong, but, I don’t believe it. The user’s inputs are so weird, and the response is so out of left field… I would put money on this being faked somehow, or there’s some missing information. Edit: Yes even with the Gemini link, I’m still suspicious. It’s just too sci-fi.

ksynwa 2y ago
https://gemini.google.com/share/6d141b742a13 https://gemini.google.com/share/6d141b742a13
crishoj 2y ago
The conversation is up on Gemini still in its entirety: https://gemini.google.com/share/6d141b742a13 https://gemini.google.com/share/6d141b742a13 Nothing out of the ordinary, except for that final response.
- Gigachad 2y ago
  The whole conversation thread is weird. But it doesn’t look like they coerced the response. It’s just so random.
  - sethammons 2y ago
    The thread didn't seam weird to me. It is someone using it for schoolwork. Some of it is an essay, some of it is bad copy-pasta multiple choice.
Pesthuf 2y ago
Sounds like they just more or less copied their homework questions and that’s why they sound so weird.
- JPLeRouzic 2y ago
  Yes, they made little effort to present the query sensibly [0]. They probably just copied/pasted homework questions even when it made no sense with aggregated words like: "truefalse". The last query before Gemini's weird answer probably aggregates two homework questions (Q15 and Q16). There is a "listen" in the query, which looks like an interjection because there probably was a button "listen" in the homework form. Overall the queries offer a somber, sinister perspective of humanity, is it surprising that it led to this kind of answer by an LLM? [0] "Nearly 10 million children in the United States live in a grandparent headed household, and of these children , around 20% are being raised without their parents in the household. Question 15 options: TrueFalse Question 16 (1 point) Listen
Timwi 2y ago
Sci-fi is probably in its training set.
exitb 2y ago
I’ve had this happen with smaller, local LLMs. It seems inspired by the fact that sometimes requests for help on the internet are met with refusals or even insults. These behaviors are mostly trained out of the big name models, but once in a while…
jfoster 2y ago
If it were fake, I don't think Google would issue this statement to CBS News: "Large language models can sometimes respond with non-sensical responses, and this is an example of that. This response violated our policies and we've taken action to prevent similar outputs from occurring." https://www.cbsnews.com/news/google-ai-chatbot-threatening-message-human-please-die/ https://www.cbsnews.com/news/google-ai-chatbot-threatening-m...
- TeMPOraL 2y ago
  It's enough for the text to appear on Gemini page for Google to issue a statement to CBS news; whether and how far out of the way did the user go to produce such a response and make it look organic, it doesn't matter - not for journalists, and thus not for Google either.
rossy 2y ago
I'm not surprised at all. LLM responses are just probability. With 100s of millions of people using LLMs daily, 1-in-a-million responses are common, so even if you haven't experienced it personally, you should expect to hear stories about wacky left field responses from LLMs. Guaranteed every LLM has tons of examples of dialogue from sci-fi "rouge AI" in its training set, and they're often told they are AI in their system prompt.
- tiahura 2y ago
  Monkeys and typewriters seems like the least likely explanation for what happened here.

ksynwa 2y ago

https://gemini.google.com/share/6d141b742a13 https://gemini.google.com/share/6d141b742a13

crishoj 2y ago

The conversation is up on Gemini still in its entirety: https://gemini.google.com/share/6d141b742a13 https://gemini.google.com/share/6d141b742a13 Nothing out of the ordinary, except for that final response.

Gigachad 2y ago
The whole conversation thread is weird. But it doesn’t look like they coerced the response. It’s just so random.
- sethammons 2y ago
  The thread didn't seam weird to me. It is someone using it for schoolwork. Some of it is an essay, some of it is bad copy-pasta multiple choice.

Pesthuf 2y ago

Sounds like they just more or less copied their homework questions and that’s why they sound so weird.

JPLeRouzic 2y ago
Yes, they made little effort to present the query sensibly [0]. They probably just copied/pasted homework questions even when it made no sense with aggregated words like: "truefalse". The last query before Gemini's weird answer probably aggregates two homework questions (Q15 and Q16). There is a "listen" in the query, which looks like an interjection because there probably was a button "listen" in the homework form. Overall the queries offer a somber, sinister perspective of humanity, is it surprising that it led to this kind of answer by an LLM? [0] "Nearly 10 million children in the United States live in a grandparent headed household, and of these children , around 20% are being raised without their parents in the household. Question 15 options: TrueFalse Question 16 (1 point) Listen

Timwi 2y ago

Sci-fi is probably in its training set.

exitb 2y ago

I’ve had this happen with smaller, local LLMs. It seems inspired by the fact that sometimes requests for help on the internet are met with refusals or even insults. These behaviors are mostly trained out of the big name models, but once in a while…

jfoster 2y ago

If it were fake, I don't think Google would issue this statement to CBS News: "Large language models can sometimes respond with non-sensical responses, and this is an example of that. This response violated our policies and we've taken action to prevent similar outputs from occurring." https://www.cbsnews.com/news/google-ai-chatbot-threatening-message-human-please-die/ https://www.cbsnews.com/news/google-ai-chatbot-threatening-m...

TeMPOraL 2y ago
It's enough for the text to appear on Gemini page for Google to issue a statement to CBS news; whether and how far out of the way did the user go to produce such a response and make it look organic, it doesn't matter - not for journalists, and thus not for Google either.

rossy 2y ago

I'm not surprised at all. LLM responses are just probability. With 100s of millions of people using LLMs daily, 1-in-a-million responses are common, so even if you haven't experienced it personally, you should expect to hear stories about wacky left field responses from LLMs. Guaranteed every LLM has tons of examples of dialogue from sci-fi "rouge AI" in its training set, and they're often told they are AI in their system prompt.

tiahura 2y ago
Monkeys and typewriters seems like the least likely explanation for what happened here.