4 ms·

Do you think they deliberately trained the model to produce images of diverse Nazis? It is clearly a byproduct of trying to correct an unaligned, bigoted model

Do you think they deliberately trained the model to produce images of diverse Nazis? It is clearly a byproduct of trying to correct an unaligned, bigoted model, and that is an example of overcorrection. > Removing bias ends with truth, not these crazy wonky results. Unfortunately there is an awful lot of untruth on the internet, if you hadn't noticed. This necessitates some correction through post-training.

margorczynski 3mo ago
> bigoted model A what? What does this even mean?
- kennywinker 3mo ago
  It means a model trained on bigoted material.
- wongarsu 3mo ago
  A model where asking for a math professor results in an image of an asian man, asking for an engineer in an image of a white man, and asking for a criminal results in an image of a black man If you try to remove that in the name of "diversity" or being "less bigoted" you quickly end up with racially diverse nazis
ibejoeb 3mo ago
That's what they did. > OpenAI invented a technique in July 2022 whereby its system would insert terms reflecting diversity (like “Black,” “female,” or “Asian”) into image-generation prompts in a way that was hidden from the user. > Google’s Gemini system seems to do something similar, taking a user’s image-generation prompt (the instruction, such as “make a painting of the founding fathers”) and inserting terms for racial and gender diversity, such as “South Asian” or “non-binary” into the prompt More links to primary sources, evidence, and official statements in the article at https://arstechnica.com/information-technology/2024/02/googles-hidden-ai-diversity-prompts-lead-to-outcry-over-historically-inaccurate-images/ https://arstechnica.com/information-technology/2024/02/googl...
- monk_grilla 3mo ago
  Lol fair enough, I stand corrected. It seems then that their objective is to superficially increase the diversity of results, to avoid bad PR, rather than actually neutralising harmful and untruthful biases of the input data.

margorczynski 3mo ago

> bigoted model A what? What does this even mean?

kennywinker 3mo ago
It means a model trained on bigoted material.
wongarsu 3mo ago
A model where asking for a math professor results in an image of an asian man, asking for an engineer in an image of a white man, and asking for a criminal results in an image of a black man If you try to remove that in the name of "diversity" or being "less bigoted" you quickly end up with racially diverse nazis

ibejoeb 3mo ago

That's what they did. > OpenAI invented a technique in July 2022 whereby its system would insert terms reflecting diversity (like “Black,” “female,” or “Asian”) into image-generation prompts in a way that was hidden from the user. > Google’s Gemini system seems to do something similar, taking a user’s image-generation prompt (the instruction, such as “make a painting of the founding fathers”) and inserting terms for racial and gender diversity, such as “South Asian” or “non-binary” into the prompt More links to primary sources, evidence, and official statements in the article at https://arstechnica.com/information-technology/2024/02/googles-hidden-ai-diversity-prompts-lead-to-outcry-over-historically-inaccurate-images/ https://arstechnica.com/information-technology/2024/02/googl...

monk_grilla 3mo ago
Lol fair enough, I stand corrected. It seems then that their objective is to superficially increase the diversity of results, to avoid bad PR, rather than actually neutralising harmful and untruthful biases of the input data.