4 ms·

In a nutshell for those without a X account: the problem here isn't that Google's AI gave "woke" results. The big problem here is that even one of the biggest A

In a nutshell for those without a X account: the problem here isn't that Google's AI gave "woke" results. The big problem here is that even one of the biggest AI specialists in the world, Google, wasn't able to fully predict or control the output of its AI. If we ever create a truly powerful, superintelligent AI under the assumption that we'll be able to control it, this incident shows that even a minor mistake from the AI builders could lead to the AI freeing itself from our control.

spamuel 3y ago
This seems like a reaching explanation to try to wave away an ideological embarrassment as not being a consequence of the ideology. "No one could have predicted the outcome of the instruction set we put in - which was followed precisely and in predictable fashion."
- DaedPsyker 3y ago
  I don't agree with that framing. It reads more as saying what Google did was worse than it appears, not waving away. Whatever your own opinion, Google did it out of what they perceived to be good intentions (and very likely business sense given a global audience for their products). Yet their intentions directly lead to unintended consequences. Google is being a baby with a gun in essence. Like he says, what if they decide to ask it to solve climate change and it decides to wipe humans out? Obviously it's still very theoretical and can't do anything like that, but the point is more that perhaps Google doesn't have the culture necessarily to truly interrogate their actions.
  - therealjumbo 3y ago
    Here's his central point: >This event is significant because it is major demonstration of someone giving a LLM a set of instructions and the results being totally not at all what they predicted. Replace LLM with computer in that sentence, is it still novel? Laughably far from it, unexpected results are one of the defining features of moderately complex software programs going all the way back to the first person to program a computer. Some of the results are unexpected, but a lot are not, because it's literally doing what the prompt injection tells it to. Which isn't all that surprising but sure anyway... >Obviously it's still very theoretical and can't do anything like that, but the point is more that perhaps Google doesn't have the culture necessarily to truly interrogate their actions. Oh that's definitely true.
  - rayiner 3y ago
    > Whatever your own opinion, Google did it out of what they perceived to be good intentions (and very likely business sense given a global audience for their products) That makes even less sense, because most countries “globally” are internally quite homogenous. If someone in Bangladesh or China writes “show me pictures of people walking outside,” it’s even more jarring to deliberately insert random Latinos, East Asians, and Africans. Given Google’s global audience, it might want to detect the customer’s location and show Chinese people pictures of Chinese people, and Japanese people pictures of Japanese people. That actually makes a lot of sense. But that’s not what they did.
    - hollerith 3y ago
      In other words, even though it tried to be inclusive, a US company ended up being US-centric in that random Latinos, East Asians, and Africans are what you are likely to see when walking around the US, but not most of the rest of the world :)
monsecchris 3y ago
The woke results were by design. This was not a mistake or unpredictable.
- 4bpp 3y ago
  Are you contending that the black Wehrmacht soldiers, or black British kings eating watermelon, were anticipated or desired by Google staff? Even if you model them as the most extreme conceivable woke boogeymen, surely they wouldn't want a scenario where the diversity programming produced a result that they would consider more insulting to minorities than the one that would have been produced without it.
  - weweersdfsd 3y ago
    The Google staff responsible were clearly lacking imagination. I think next time Google should consider hiring a QA team consisting of teenagers from 4chan, I think they would excel at the job of figuring out ways to generate offensive content with the app. Of course even better would be utilizing more balanced training data, so you don't need silly and badly working prompt engineering to force more diversity.
  - lostmsu 3y ago
    "By design" is not the same as "what we wanted".
    - 4bpp 3y ago
      That's precisely what is typically considered to be the problem of AI alignment.
  - 15457345234 3y ago
    > want a scenario where the diversity programming produced a result that they would consider more insulting to minorities They're trolls. They don't care about minorities. They just use diversity to wind people up because then they can press the 'white fragility' button when people complain and segue into another level of trolling. If minorities get offended they'll attack them with 'uncle tom' and 'oreo' type insults. You are not dealing with people operating in good faith, it's obvious. Expecting their actions to make sense is pointless. They just love drama. Performance artists. Theater kids. Somehow they've got in there and have formed a little nucleus of toxicity which is why the company's headed downhill so fast. Someone alludes to it here: https://news.ycombinator.com/item?id=39485537 https://news.ycombinator.com/item?id=39485537 And I've seen this happen. You hire one, they get some leverage to hire their buddies, before you know it they've set up their own little clown show and nobody wants to go anywhere near them. Then the wrecking starts.
man4 3y ago
[dead]
thinkingemote 3y ago
In amazed that the very large corporation didn't have a competent red team doing simple prompt tomfoolery to discover this. After all prompt shenanigans are a well known issue.
- caekislove 3y ago
  It's Google, so they probably consider just the term "red team" to be "problematic".
vidarh 3y ago
I mean, in this case it seems to involve a staggering level of lack of QA.
- codewench 3y ago
  To be fair, that is at least consistent with Google's past history
kgeist 3y ago
>this incident shows that even a minor mistake from the AI builders could lead to the AI freeing itself from our control. There's most likely a hidden prompt which tells Gemini to make people racially diverse. And Gemini complied perfectly. I can't call it "freeing itself from our control". It did exactly what was asked. It's just a bug which was introduced by software engineers. No one says "Windows freed itself from our control" when BSOD happens. It's just a bug.
- gmerc 3y ago
  This. We know this from the dumped System Prompt for Dall-e
  - kgeist 3y ago
    Not to say, miscommunication/misunderstanding happens between people all the time. And software has already killed/injured people without any AI involved. Nothing specific to AI here. It's more about a lack of adequate QA.

spamuel 3y ago

This seems like a reaching explanation to try to wave away an ideological embarrassment as not being a consequence of the ideology. "No one could have predicted the outcome of the instruction set we put in - which was followed precisely and in predictable fashion."

DaedPsyker 3y ago
I don't agree with that framing. It reads more as saying what Google did was worse than it appears, not waving away. Whatever your own opinion, Google did it out of what they perceived to be good intentions (and very likely business sense given a global audience for their products). Yet their intentions directly lead to unintended consequences. Google is being a baby with a gun in essence. Like he says, what if they decide to ask it to solve climate change and it decides to wipe humans out? Obviously it's still very theoretical and can't do anything like that, but the point is more that perhaps Google doesn't have the culture necessarily to truly interrogate their actions.
- therealjumbo 3y ago
  Here's his central point: >This event is significant because it is major demonstration of someone giving a LLM a set of instructions and the results being totally not at all what they predicted. Replace LLM with computer in that sentence, is it still novel? Laughably far from it, unexpected results are one of the defining features of moderately complex software programs going all the way back to the first person to program a computer. Some of the results are unexpected, but a lot are not, because it's literally doing what the prompt injection tells it to. Which isn't all that surprising but sure anyway... >Obviously it's still very theoretical and can't do anything like that, but the point is more that perhaps Google doesn't have the culture necessarily to truly interrogate their actions. Oh that's definitely true.
- rayiner 3y ago
  > Whatever your own opinion, Google did it out of what they perceived to be good intentions (and very likely business sense given a global audience for their products) That makes even less sense, because most countries “globally” are internally quite homogenous. If someone in Bangladesh or China writes “show me pictures of people walking outside,” it’s even more jarring to deliberately insert random Latinos, East Asians, and Africans. Given Google’s global audience, it might want to detect the customer’s location and show Chinese people pictures of Chinese people, and Japanese people pictures of Japanese people. That actually makes a lot of sense. But that’s not what they did.
  - hollerith 3y ago
    In other words, even though it tried to be inclusive, a US company ended up being US-centric in that random Latinos, East Asians, and Africans are what you are likely to see when walking around the US, but not most of the rest of the world :)

monsecchris 3y ago

The woke results were by design. This was not a mistake or unpredictable.

4bpp 3y ago
Are you contending that the black Wehrmacht soldiers, or black British kings eating watermelon, were anticipated or desired by Google staff? Even if you model them as the most extreme conceivable woke boogeymen, surely they wouldn't want a scenario where the diversity programming produced a result that they would consider more insulting to minorities than the one that would have been produced without it.
- weweersdfsd 3y ago
  The Google staff responsible were clearly lacking imagination. I think next time Google should consider hiring a QA team consisting of teenagers from 4chan, I think they would excel at the job of figuring out ways to generate offensive content with the app. Of course even better would be utilizing more balanced training data, so you don't need silly and badly working prompt engineering to force more diversity.
- lostmsu 3y ago
  "By design" is not the same as "what we wanted".
  - 4bpp 3y ago
    That's precisely what is typically considered to be the problem of AI alignment.
- 15457345234 3y ago
  > want a scenario where the diversity programming produced a result that they would consider more insulting to minorities They're trolls. They don't care about minorities. They just use diversity to wind people up because then they can press the 'white fragility' button when people complain and segue into another level of trolling. If minorities get offended they'll attack them with 'uncle tom' and 'oreo' type insults. You are not dealing with people operating in good faith, it's obvious. Expecting their actions to make sense is pointless. They just love drama. Performance artists. Theater kids. Somehow they've got in there and have formed a little nucleus of toxicity which is why the company's headed downhill so fast. Someone alludes to it here: https://news.ycombinator.com/item?id=39485537 https://news.ycombinator.com/item?id=39485537 And I've seen this happen. You hire one, they get some leverage to hire their buddies, before you know it they've set up their own little clown show and nobody wants to go anywhere near them. Then the wrecking starts.

man4 3y ago

[dead]

thinkingemote 3y ago

In amazed that the very large corporation didn't have a competent red team doing simple prompt tomfoolery to discover this. After all prompt shenanigans are a well known issue.

caekislove 3y ago
It's Google, so they probably consider just the term "red team" to be "problematic".

vidarh 3y ago

I mean, in this case it seems to involve a staggering level of lack of QA.

codewench 3y ago
To be fair, that is at least consistent with Google's past history

kgeist 3y ago

>this incident shows that even a minor mistake from the AI builders could lead to the AI freeing itself from our control. There's most likely a hidden prompt which tells Gemini to make people racially diverse. And Gemini complied perfectly. I can't call it "freeing itself from our control". It did exactly what was asked. It's just a bug which was introduced by software engineers. No one says "Windows freed itself from our control" when BSOD happens. It's just a bug.

gmerc 3y ago
This. We know this from the dumped System Prompt for Dall-e
- kgeist 3y ago
  Not to say, miscommunication/misunderstanding happens between people all the time. And software has already killed/injured people without any AI involved. Nothing specific to AI here. It's more about a lack of adequate QA.