4 ms·

gemini seems to have problems generating white people and honestly this just opens the door for things that are even more racist [1], the harder you try the mor

gemini seems to have problems generating white people and honestly this just opens the door for things that are even more racist [1], the harder you try the more you'll fail, just get over the DEI nonsense already 1. https://twitter.com/wagieeacc/status/1760371304425762940 https://twitter.com/wagieeacc/status/1760371304425762940

Jason_Protell 3y ago
Is there any evidence that this is a consequence of DEI rather than a deeper technical issue?
- Jensson 3y ago
  You get 4 images per time and are lucky to get one white person when asked for it, no other model has that issue. Other models has no problems generating black people either, so it isn't that other models only generates white people. So either it isn't a technical issue or Google failed to solve a problem everyone else easily solved. The chances of this having nothing to do with DEI is basically 0.
  - ceejayoz 3y ago
    Depending on how broadly you define it, something like 10-30% of the world's population is white. Africa is about 20% of the world population; Asia is 60% of it. One in four sounds about right?
    - cm2012 3y ago
      It does the same if you ask for pictures of past popes, 1945 German soldiers, etc.
      - ceejayoz 3y ago
        It'll also add extra fingers to human hands. Presumably that's not because of DEI guardrails about polydactyly, right? The current state of the art in AI gets things wrong regularly.
        cm2012 3y ago
        Sure, but this one is from Google adding a tag to make every image of people diverse, not AI randomness.
        ceejayoz 3y ago
        Am I missing something in the link demonstrating that, or is it conjecture?
        cm2012 3y ago
        It's been demonstrated on Twitter a few times, can't find a link handy
        perlclutcher 3y ago
        https://twitter.com/altryne/status/1760358916624719938 https://twitter.com/altryne/status/1760358916624719938 Here's some corporate-lawyer-speak straight from Google: > We are aware that Gemini is offering inaccuracies... > As part of our AI principles, we design our image generation capabilities to reflect our global user base, and we take representation and bias seriously.
        deleted 3y ago
        [deleted]
        ceejayoz 3y ago
        That doesn't back up the assertion; it's easily read as "we make sure our training sets reflect the 85% of the world that doesn't live in Europe and North America". Again, 1/4 white people is statistically what you'd expect.
    - wruza 3y ago
      If it does, shouldn't there be 60% asians?
- 123yawaworht456 3y ago
  yes, there's irrefutable evidence that models are wrangled into abiding the commissars' vision rather than just do their job and output the product of their training data. https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf
- nickthegreek 3y ago
  I dont think so. My boss wanted me to generate a birthday image for a co-worker of a John Cena flyfishing. ChatGPT refused to do so. So I had to move to describing the type of person John Cena is instead of using his name. I kept giving me bearded people no matter what. I thought this would be the perfect time to try out Gemini for the first time. Well shit, It wont even give me a white guy. But all the black dudes are beardless. update: google agrees there is an issue. https://news.ycombinator.com/item?id=39459270 https://news.ycombinator.com/item?id=39459270
  - 8f2ab37a-ed6c 3y ago
    It feels that the image generation it offers is perfect for some sort of a California-Corporate Style, e.g. you ask it for a "photo of people at the board room" or "people at the company cafeteria" and you get the corporate friendly ratio of colors, ability-levels, sizes etc. See Google's various image assets: https://www.google.com/about/careers/applications/ https://www.google.com/about/careers/applications/ . It's great for coastal and urban marketing brochures. But then then same California Corporate style makes no sense for historical images, so perhaps this is where Midjourney comes in.
- minimaxir 3y ago
  When DALL-E 2 was released in 2022, OpenAI published an article noting that the inclusion of guardrails was a correction for bias: https://openai.com/blog/reducing-bias-and-improving-safety-in-dall-e-2 https://openai.com/blog/reducing-bias-and-improving-safety-i... It was widely criticized back then: the fact that Google both brought it back and made it more prominent is weird. Notably, OpenAI's implementation is more scoped.
- mike_d 3y ago
  It is possible Google tried to avoid likenesses of well known people by removing any image from the training data that contained a face and then including a controlled set of people images. If you give a contractor a project that you want 200k images of people who are not famous, they will send teams to regions where you may only have to pay each person a few dollars to be photographed. Likely SE Asia and Africa.
- allmadhare22 3y ago
  Depending on what you ask for, it injects the word 'diverse' into the response description, so it's pretty obvious they're brute forcing diversity into it. E.g. "Generate me an image of a family" and you will get back "Here are some images of a diverse family".
- sotasota 3y ago
  https://pbs.twimg.com/media/GG1eyKjXQAA1FxU?format=jpg&name=large https://pbs.twimg.com/media/GG1eyKjXQAA1FxU?format=jpg&name=... https://cdn.sanity.io/images/cjtc1tnd/production/912b6b5aaccb7955fb5ce48c1f055b4d579224dd-1156x1058.png?w=1156&auto=format https://cdn.sanity.io/images/cjtc1tnd/production/912b6b5aacc... https://pbs.twimg.com/media/GG1ThfsWUAAp-SO?format=jpg&name=large https://pbs.twimg.com/media/GG1ThfsWUAAp-SO?format=jpg&name=... https://cdn.sanity.io/images/cjtc1tnd/production/e2810c02ff6ac0033bccda8e8d2a07f4935c0d7e-2232x962.png?w=2000&auto=format https://cdn.sanity.io/images/cjtc1tnd/production/e2810c02ff6... https://pbs.twimg.com/media/GG1MnepXwAAkPL6?format=jpg&name=large https://pbs.twimg.com/media/GG1MnepXwAAkPL6?format=jpg&name=... https://pbs.twimg.com/media/GG0BLVsbMAARZXr?format=jpg&name=medium https://pbs.twimg.com/media/GG0BLVsbMAARZXr?format=jpg&name=...
  - flumpcakes 3y ago
    I don't understand how people could even argue that this is in any way acceptable. Fighting "bias" has become some boogyman and anything "non-white" is now beyond reproach. Shocking.
    - gedy 3y ago
      Seriously, I've basically written off using Gemini for good after this HR style nonsense. It's a shame that Google, who invented much of this tech, is so crippled by their own people's politics.
    - AnarchismIsCool 3y ago
      Fighting bias is a good thing, you'd have to be pretty...er...biased to believe otherwise. Bias is fundamentally a distortion or deviation from objective reality. This, on the other hand, is just fucking stupid political showboating that's hurting their SV white knight cause. It's just differently flavored bias
  - gs17 3y ago
    "I can't generate white British royalty because they exist, but I can make up black ones" is pretty close to an actually valid reason.
AnarchismIsCool 3y ago
I don't think the DEI stuff is nonsense, but SV is sensitive to this because most of their previous generation of models were horrifyingly racist if not teenage nazis, and so they turned the anti-racism knob up to 11 which made the models....racist but in a different way. Like depicting colonial settlers as native americans is extremely problematic in its own special way, but I also don't expect a statistical solver to grasp that context meaningfully.
- grotorea 3y ago
  So you're saying in a way this is /pol/ and Tay's fault?
  - AnarchismIsCool 3y ago
    Looks around at everything ...is there anything that isn't 4chan's fault at this point? Realistically, kinda. There have always been tons of anecdotes of video conference systems not following black people, cameras not white balancing correctly on darker faces etc. That era of SV was plagued by systems that were built by a bunch of young white guys who never tested them with anyone else. I'm not saying they were inherently racist or anything, just that the broader society really lambasted them for it and so they attempted to correct. Really, the pendulum will continue to swing and we'll see it eventually center up on something approaching sanity but the hyper-authoritarian sentiment that SV seems to have (we're geniuses and the public is stupid, we need to correct them) is...a troubling direction.
    - callalex 3y ago
      I’m always hesitant to jump straight to ringing the racism bell when it comes to problems that arise from fundamental physics. I have dark skin, and I constantly struggle with automatic faucets, soap dispensers, and towel dispensers in public bathrooms. None of those are made by big tech.
xdennis 3y ago
It's not just Gemini, it's Google. And old example is to just search "white people" on Google Images. Almost all the results are black people. https://www.google.com/search?q=white+people&tbm=isch&hl=ro https://www.google.com/search?q=white+people&tbm=isch&hl=ro

Jason_Protell 3y ago

Is there any evidence that this is a consequence of DEI rather than a deeper technical issue?

Jensson 3y ago
You get 4 images per time and are lucky to get one white person when asked for it, no other model has that issue. Other models has no problems generating black people either, so it isn't that other models only generates white people. So either it isn't a technical issue or Google failed to solve a problem everyone else easily solved. The chances of this having nothing to do with DEI is basically 0.
- ceejayoz 3y ago
  Depending on how broadly you define it, something like 10-30% of the world's population is white. Africa is about 20% of the world population; Asia is 60% of it. One in four sounds about right?
  - cm2012 3y ago
    It does the same if you ask for pictures of past popes, 1945 German soldiers, etc.
    - ceejayoz 3y ago
      It'll also add extra fingers to human hands. Presumably that's not because of DEI guardrails about polydactyly, right? The current state of the art in AI gets things wrong regularly.
      - cm2012 3y ago
        Sure, but this one is from Google adding a tag to make every image of people diverse, not AI randomness.
        ceejayoz 3y ago
        Am I missing something in the link demonstrating that, or is it conjecture?
        cm2012 3y ago
        It's been demonstrated on Twitter a few times, can't find a link handy
        perlclutcher 3y ago
        https://twitter.com/altryne/status/1760358916624719938 https://twitter.com/altryne/status/1760358916624719938 Here's some corporate-lawyer-speak straight from Google: > We are aware that Gemini is offering inaccuracies... > As part of our AI principles, we design our image generation capabilities to reflect our global user base, and we take representation and bias seriously.
        deleted 3y ago
        [deleted]
        ceejayoz 3y ago
        That doesn't back up the assertion; it's easily read as "we make sure our training sets reflect the 85% of the world that doesn't live in Europe and North America". Again, 1/4 white people is statistically what you'd expect.
  - wruza 3y ago
    If it does, shouldn't there be 60% asians?
123yawaworht456 3y ago
yes, there's irrefutable evidence that models are wrangled into abiding the commissars' vision rather than just do their job and output the product of their training data. https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf
nickthegreek 3y ago
I dont think so. My boss wanted me to generate a birthday image for a co-worker of a John Cena flyfishing. ChatGPT refused to do so. So I had to move to describing the type of person John Cena is instead of using his name. I kept giving me bearded people no matter what. I thought this would be the perfect time to try out Gemini for the first time. Well shit, It wont even give me a white guy. But all the black dudes are beardless. update: google agrees there is an issue. https://news.ycombinator.com/item?id=39459270 https://news.ycombinator.com/item?id=39459270
- 8f2ab37a-ed6c 3y ago
  It feels that the image generation it offers is perfect for some sort of a California-Corporate Style, e.g. you ask it for a "photo of people at the board room" or "people at the company cafeteria" and you get the corporate friendly ratio of colors, ability-levels, sizes etc. See Google's various image assets: https://www.google.com/about/careers/applications/ https://www.google.com/about/careers/applications/ . It's great for coastal and urban marketing brochures. But then then same California Corporate style makes no sense for historical images, so perhaps this is where Midjourney comes in.
minimaxir 3y ago
When DALL-E 2 was released in 2022, OpenAI published an article noting that the inclusion of guardrails was a correction for bias: https://openai.com/blog/reducing-bias-and-improving-safety-in-dall-e-2 https://openai.com/blog/reducing-bias-and-improving-safety-i... It was widely criticized back then: the fact that Google both brought it back and made it more prominent is weird. Notably, OpenAI's implementation is more scoped.
mike_d 3y ago
It is possible Google tried to avoid likenesses of well known people by removing any image from the training data that contained a face and then including a controlled set of people images. If you give a contractor a project that you want 200k images of people who are not famous, they will send teams to regions where you may only have to pay each person a few dollars to be photographed. Likely SE Asia and Africa.
allmadhare22 3y ago
Depending on what you ask for, it injects the word 'diverse' into the response description, so it's pretty obvious they're brute forcing diversity into it. E.g. "Generate me an image of a family" and you will get back "Here are some images of a diverse family".
sotasota 3y ago
https://pbs.twimg.com/media/GG1eyKjXQAA1FxU?format=jpg&name=large https://pbs.twimg.com/media/GG1eyKjXQAA1FxU?format=jpg&name=... https://cdn.sanity.io/images/cjtc1tnd/production/912b6b5aaccb7955fb5ce48c1f055b4d579224dd-1156x1058.png?w=1156&auto=format https://cdn.sanity.io/images/cjtc1tnd/production/912b6b5aacc... https://pbs.twimg.com/media/GG1ThfsWUAAp-SO?format=jpg&name=large https://pbs.twimg.com/media/GG1ThfsWUAAp-SO?format=jpg&name=... https://cdn.sanity.io/images/cjtc1tnd/production/e2810c02ff6ac0033bccda8e8d2a07f4935c0d7e-2232x962.png?w=2000&auto=format https://cdn.sanity.io/images/cjtc1tnd/production/e2810c02ff6... https://pbs.twimg.com/media/GG1MnepXwAAkPL6?format=jpg&name=large https://pbs.twimg.com/media/GG1MnepXwAAkPL6?format=jpg&name=... https://pbs.twimg.com/media/GG0BLVsbMAARZXr?format=jpg&name=medium https://pbs.twimg.com/media/GG0BLVsbMAARZXr?format=jpg&name=...
- flumpcakes 3y ago
  I don't understand how people could even argue that this is in any way acceptable. Fighting "bias" has become some boogyman and anything "non-white" is now beyond reproach. Shocking.
  - gedy 3y ago
    Seriously, I've basically written off using Gemini for good after this HR style nonsense. It's a shame that Google, who invented much of this tech, is so crippled by their own people's politics.
  - AnarchismIsCool 3y ago
    Fighting bias is a good thing, you'd have to be pretty...er...biased to believe otherwise. Bias is fundamentally a distortion or deviation from objective reality. This, on the other hand, is just fucking stupid political showboating that's hurting their SV white knight cause. It's just differently flavored bias
- gs17 3y ago
  "I can't generate white British royalty because they exist, but I can make up black ones" is pretty close to an actually valid reason.

AnarchismIsCool 3y ago

I don't think the DEI stuff is nonsense, but SV is sensitive to this because most of their previous generation of models were horrifyingly racist if not teenage nazis, and so they turned the anti-racism knob up to 11 which made the models....racist but in a different way. Like depicting colonial settlers as native americans is extremely problematic in its own special way, but I also don't expect a statistical solver to grasp that context meaningfully.

grotorea 3y ago
So you're saying in a way this is /pol/ and Tay's fault?
- AnarchismIsCool 3y ago
  Looks around at everything ...is there anything that isn't 4chan's fault at this point? Realistically, kinda. There have always been tons of anecdotes of video conference systems not following black people, cameras not white balancing correctly on darker faces etc. That era of SV was plagued by systems that were built by a bunch of young white guys who never tested them with anyone else. I'm not saying they were inherently racist or anything, just that the broader society really lambasted them for it and so they attempted to correct. Really, the pendulum will continue to swing and we'll see it eventually center up on something approaching sanity but the hyper-authoritarian sentiment that SV seems to have (we're geniuses and the public is stupid, we need to correct them) is...a troubling direction.
  - callalex 3y ago
    I’m always hesitant to jump straight to ringing the racism bell when it comes to problems that arise from fundamental physics. I have dark skin, and I constantly struggle with automatic faucets, soap dispensers, and towel dispensers in public bathrooms. None of those are made by big tech.

xdennis 3y ago

It's not just Gemini, it's Google. And old example is to just search "white people" on Google Images. Almost all the results are black people. https://www.google.com/search?q=white+people&tbm=isch&hl=ro https://www.google.com/search?q=white+people&tbm=isch&hl=ro