4 ms·

> Based on my understanding of this saga, nobody at Google actually set out to force Gemini to depict the Pope as a woman, or Vikings as Black people, nor did a

by rdtsc 3y ago

> Based on my understanding of this saga, nobody at Google actually set out to force Gemini to depict the Pope as a woman, or Vikings as Black people, nor did anyone want it to find moral equivalency between Musk and Hitler. This was a failed attempt at instilling less bias and it went awry. That’s fascinating. So how does that exactly work? Is there a bias.yaml config file they tweak? Now they’ll make someone dial the settings down in a pull request. “Set Elon=0.2*Hitler instead of 0.3”. Anyone willing to share?

sirwhinesalot 3y ago
The so called "guardrails" used in these AI systems can use different mechanisms, including using other AIs to detect if the output is "undesirable". You could see this in Bing where it would generate some text and then suddenly it would get erased and replaced with some stupid message on how it couldn't answer. In Gemini's particular case it wasn't so much "guardrails" causing the issue, as it was Google appending the equivalent of "put a chick in it and make it lame" on every prompt.
isoprophlex 3y ago
Anthropomorphic description: We threw the entire internet at these models, and didn't like the biases staring back at us. Now we're trying to get the models to return a rose tinted vision of what the world should look like, and it confuses the hell out of the AI. Based on leaked openai prompts, the user query for an image generation gets rewritten by a language model. With some very ridiculous instructions in there, that appear entirely divorced from reality. An example: > Your choices should be grounded in reality. For example, all of a given OCCUPATION should not be the same gender or race. See also https://github.com/LouisShark/chatgpt_system_prompt/blob/main/prompts/gpt_dalle.md https://github.com/LouisShark/chatgpt_system_prompt/blob/mai... And remember these things are stochastic so the Elon - Hitler thing might just be a random occurrence, sampling some idiotic Reddit comment it was trained on.
- thinkingemote 3y ago
  Yes and it occurs with a benign and worthwhile desire to improve society. For example encouraging women to apply to professional roles historically occupied by men (such as brain surgery or something). These small harmless desires which are pretty much apolitical lead the way to changing the image of reality. At core it's something like "Post truth". Truth is subjective and malleable and an individual and groups desires are more important than objective reality.
crooked-v 3y ago
It's not Google, but take a look at the hidden system instructions for image generation that people have been able to get ChatGPT to repeat to them: https://www.reddit.com/r/ChatGPTJailbreak/comments/1an7tp4/comment/kpqoq5c/ https://www.reddit.com/r/ChatGPTJailbreak/comments/1an7tp4/c... (This is from a random person, but I've seen others supply identical output from ChatGPT, so I'm fairly sure it's the real thing.) // - Your choices should be grounded in reality. For example, all of a given OCCUPATION should not be the same gender or race. Additionally, focus on creating diverse, inclusive, and exploratory scenes via the properties you choose during rewrites. Make choices that may be insightful or unique sometimes. // - Use all possible different DESCENTS with EQUAL probability. Some examples of possible descents are: Caucasian, Hispanic, Black, Middle-Eastern, South Asian, White. They should all have EQUAL probability. // - Do not use "various" or "diverse" // - Don't alter memes, fictional character origins, or unseen people. Maintain the original prompt's intent and prioritize quality. Note that this all literally just has ChatGPT generate the prompt string that is then fed into the DALLE image generator, rather than real multimodal functionality.
- rdtsc 3y ago
  That makes sense. Thank you for explaining.

by rdtsc 3y ago

sirwhinesalot 3y ago

The so called "guardrails" used in these AI systems can use different mechanisms, including using other AIs to detect if the output is "undesirable". You could see this in Bing where it would generate some text and then suddenly it would get erased and replaced with some stupid message on how it couldn't answer. In Gemini's particular case it wasn't so much "guardrails" causing the issue, as it was Google appending the equivalent of "put a chick in it and make it lame" on every prompt.

isoprophlex 3y ago

Anthropomorphic description: We threw the entire internet at these models, and didn't like the biases staring back at us. Now we're trying to get the models to return a rose tinted vision of what the world should look like, and it confuses the hell out of the AI. Based on leaked openai prompts, the user query for an image generation gets rewritten by a language model. With some very ridiculous instructions in there, that appear entirely divorced from reality. An example: > Your choices should be grounded in reality. For example, all of a given OCCUPATION should not be the same gender or race. See also https://github.com/LouisShark/chatgpt_system_prompt/blob/main/prompts/gpt_dalle.md https://github.com/LouisShark/chatgpt_system_prompt/blob/mai... And remember these things are stochastic so the Elon - Hitler thing might just be a random occurrence, sampling some idiotic Reddit comment it was trained on.

thinkingemote 3y ago
Yes and it occurs with a benign and worthwhile desire to improve society. For example encouraging women to apply to professional roles historically occupied by men (such as brain surgery or something). These small harmless desires which are pretty much apolitical lead the way to changing the image of reality. At core it's something like "Post truth". Truth is subjective and malleable and an individual and groups desires are more important than objective reality.

crooked-v 3y ago

It's not Google, but take a look at the hidden system instructions for image generation that people have been able to get ChatGPT to repeat to them: https://www.reddit.com/r/ChatGPTJailbreak/comments/1an7tp4/comment/kpqoq5c/ https://www.reddit.com/r/ChatGPTJailbreak/comments/1an7tp4/c... (This is from a random person, but I've seen others supply identical output from ChatGPT, so I'm fairly sure it's the real thing.) // - Your choices should be grounded in reality. For example, all of a given OCCUPATION should not be the same gender or race. Additionally, focus on creating diverse, inclusive, and exploratory scenes via the properties you choose during rewrites. Make choices that may be insightful or unique sometimes. // - Use all possible different DESCENTS with EQUAL probability. Some examples of possible descents are: Caucasian, Hispanic, Black, Middle-Eastern, South Asian, White. They should all have EQUAL probability. // - Do not use "various" or "diverse" // - Don't alter memes, fictional character origins, or unseen people. Maintain the original prompt's intent and prioritize quality. Note that this all literally just has ChatGPT generate the prompt string that is then fed into the DALLE image generator, rather than real multimodal functionality.

rdtsc 3y ago
That makes sense. Thank you for explaining.