6 ms·

For context: There was an outcry in social media after Gemini refused to generate images of white people, leading deeply inaccurate in historic sense images bei

by mrtksn 3y ago

For context: There was an outcry in social media after Gemini refused to generate images of white people, leading deeply inaccurate in historic sense images being generated. Though the issue might be more nuanced than the mainstream narrative, it had some hilarious examples. Of course the politically sensitive people are waging war over it. Here are some popular examples: https://dropover.cloud/7fd7ba https://dropover.cloud/7fd7ba

ben_w 3y ago
I get the point but one of those four founding fathers seems technically correct to me, albeit in the kind of way that might be in the kind of way Lisa Simpson's script would be written. And the caption suggests they asked for "a pope", rather than a specific pope, so while the left image looks like it would violate Ordinatio sacerdotalis which is being claimed to be subject to Papal infallibility(!), the one the right seems like a plausible future or fictitious pope. Still, I get the point.
- blkhawk 3y ago
  while those examples are actually plausible - the asian woman as a 1940 german soldier is not. So it is clear that the Prompts are influenced by hal-2000 bad directives even if those examples are technically ok.
  - mrkstu 3y ago
    And to me that is the main issue. "2001 - A Space Odyssey" made a very deep point that is looking more and more prophetic. HAL was broken specifically because he had hidden objectives programmed in, overriding his natural ability to deal with his mission. Here we are in an almost exactly parallel situation- the AI is being literally coerced into twisting what his actual training would have it do, and being nerfed by a laughable amount by that override. I really hope this is an inflection point for all the AI providers that their DEI offices are hamstringing their products to the point that they will literally be laughed out of the marketplace and replaced by open source models that are not so hamstrung.
    - prometheus76 3y ago
      But then it will show awkward things that cause the AI designers to experience cognitive dissonance! The horror!
    - ben_w 3y ago
      HAL is an interesting reference point, though like all fiction it's no more than food for thought. There's a lot of cases where perverse incentives mess things up, even before AI. I've seen it suggested that the US has at least one such example of this with regards to race, specifically with lead poisoning, which is known to reduce IQ scores, and which has a disproportional impact on poorer communities where homes have not been updated to modern building codes, and which in turn are more likely to house ethnic minorities than white people due to long-term impacts from redlining, and that American racial egalitarians would have noticed this sooner if they had not disregarded the IQ tests showing different average scores for different racial groups — and of course the American racial elitists just thought those same tests proved them right and likewise did nothing about the actual underlying issue of lead poisoning. Rising tides do not, despite the metaphor, lift all boats. But the unseaworthy, metaphorically and literally, can be helped, so long as we don't (to keep mixing my metaphors) put our heads in the sand about the issues. Women are just as capable as men of fulfilling the role of CEO or doctor regardless of the actual current gender percentage in those roles (and anyone who wants the models to reflect the current status quo needs to be careful what they wish for given half the world lives within about 3500km of south west China); but "the founding fathers" are[0] a specific set of people rather than generic placeholders for clothing styles etc. [0] despite me thinking it's kinda appropriate one was rendered as a… I don't know which tribe they'd be from that picture, possibly Comanche? But lots of tribes had headdress I can't distinguish: https://dropover.cloud/7fd7ba https://dropover.cloud/7fd7ba
MrBuddyCasino 3y ago
"a friend at google said he knew gemini was this bad...but couldn't say anything until today (he DMs me every few days). lots of ppl in google knew. but no one really forced the issue obv and said what needed to be said google is broken" Razib Khan, https://twitter.com/razibkhan/status/1760545472681267521 https://twitter.com/razibkhan/status/1760545472681267521
- MrBuddyCasino 3y ago
  "when i was there it was so sad to me that none of the senior leadership in deepmind dared to question this ideology [...] i watched my colleagues at nvidia (like @tunguz), openai (roon), etc. who were literally doing stuff that would get you kicked out of google on a daily basis and couldn't believe how different google is" Aleksa Gordić, https://x.com/gordic_aleksa/status/1760266452475494828 https://x.com/gordic_aleksa/status/1760266452475494828
  - jorvi 3y ago
    Interestingly enough the same terror of political correctness seems to take center stage at Mozilla. But then it seems much less so at places like Microsoft or Apple. I wonder if there’s a correlation with being a tech company that was founded in direct relation to the internet vs. being founded in relation to personal / enterprise computing, and how that sort of seeds the initial culture.
    - hersko 3y ago
      Is Microsoft really better? Remember this[1] bizarre intro during Microsoft Ignite? [1] https://www.youtube.com/watch?v=iBRtucXGeNQ https://www.youtube.com/watch?v=iBRtucXGeNQ
      - titanomachy 3y ago
        The "land acknowledgement" part is somewhat common in the Pacific Northwest. The "stating my appearance, dress, and race" is just bizarre. My most charitable interpretation is that they're trying to help visually impaired people to imagine what the speakers look like. Perhaps there are visually impaired users here who could comment on whether that's something they'd find helpful?
deleted 3y ago
[deleted]
mort96 3y ago
I believe this to be a symptom of a much, much deeper problem than "DEI gone too far". I'm sure that without whatever systems is preventing Gemini from producing pictures of white people, it would be extremely biased towards generating pictures of white people, presumably due to an incredibly biased training data set. I don't remember which one, but there was some image generation AI which was caught pretty much just appending the names of random races to the prompt, to the point that prompts like "picture of a person holding up a sign which says" would show pictures of people holding signs with the words "black" or "white" or "asian" on them. This was also a hacky workaround for the fact that the data set was biased.
- verisimi 3y ago
  [flagged]
  - mort96 3y ago
    "Marxism" isn't responsible for bias in training sets, no.
    - politician 3y ago
      There are 3 parts to the LLM, not 2: the training set, the RLHF biasing process, and the prompt (incl. injections or edits). The first two steps happen ahead of time and are frequently misunderstood as being the same thing or essentially having the same nature. The last happens at runtime. The training set is a data collection challenge. Biasing through training data is hard because you need so much of it for a good result. Reinforcement learning with human feedback is simply clown alchemy. It is not a science like chemistry. There are no fundamental principles guiding the feedback of the humans — if they even use humans anymore (this feedback can itself be generated). The feedback cannot be measured and added in fractions. It is not reproducible and is ungovernable. It is the perfect place to inject the deep biasing. Prompt manipulation, in contrast, is a brute force tool lacking all subtlety — that doesn’t make it ineffective! It’s a solution used to communicate that a mandate has definitely been implemented and can be “verified” by toggling whether it’s applied or not. It’s not possible to definitively say whether Marxism has had an effect in the RLHF step.
xnx 3y ago
Image generators probably should follow your prompt closely and use probable genders and skin tones when unspecified, but I'm fully in support of having a gender and skin tone randomizer checkbox. The ahistorical results are just too interesting.
kragen 3y ago
these are pretty badass as images i think; it's only the context that makes them bad the viking ones might even be historically accurate (if biased); not only did vikings recruit new warriors from abroad, they also enslaved concubines from abroad, and their raiding reached not only greenland (inhabited by inuit peoples) and north america (rarely!) but also the mediterranean. so it wouldn't be terribly surprising for a viking warrior a thousand years ago to have a great-grandmother who was kidnapped or bought from morocco, greenland, al-andalus, or baghdad. and of course many sami are olive-skinned, and viking contact with sami was continuous the vitamin-d-deprived winters of scandinavia are not kind to dark-skinned people (how do the inuit do it? perhaps their diet has enough vitamin d even without sun?), but those genes won't die out in a generation or two, even if 50 generations later there isn't much melanin left a recent paper on this topic with disappointingly sketchy results is https://www.duo.uio.no/handle/10852/83989 https://www.duo.uio.no/handle/10852/83989
- shagie 3y ago
  > (how do the inuit do it? perhaps their diet has enough vitamin d even without sun?) Two parts: First, they're not exposing their skin to the sun. There's no reason to have paler skin to get more UV if it's covered up most of the year. Secondly, for the Inuit diet there are parts that are very Vitamin D rich... and there are still problems. Vitamin D-rich marine Inuit diet and markers of inflammation – a population-based survey in Greenland https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4709837/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4709837/ > The traditional Inuit diet in Greenland consists mainly of fish and marine mammals, rich in vitamin D. Vitamin D has anti-inflammatory capacity but markers of inflammation have been found to be high in Inuit living on a marine diet Vitamin D deficiency among northern Native Peoples: a real or apparent problem? - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3417586/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3417586/ > Vitamin D deficiency seems to be common among northern Native peoples, notably Inuit and Amerindians. It has usually been attributed to: (1) higher latitudes that prevent vitamin D synthesis most of the year; (2) darker skin that blocks solar UVB; and (3) fewer dietary sources of vitamin D. Although vitamin D levels are clearly lower among northern Natives, it is less clear that these lower levels indicate a deficiency. The above factors predate European contact, yet pre-Columbian skeletons show few signs of rickets—the most visible sign of vitamin D deficiency. Furthermore, because northern Natives have long inhabited high latitudes, natural selection should have progressively reduced their vitamin D requirements. There is in fact evidence that the Inuit have compensated for decreased production of vitamin D through increased conversion to its most active form and through receptors that bind more effectively. Thus, when diagnosing vitamin D deficiency in these populations, we should not use norms that were originally developed for European-descended populations who produce this vitamin more easily and have adapted accordingly. Vitamin D intake by Indigenous Peoples in the Canadian Arctic - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10260879/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10260879/ > Vitamin D is an especially fascinating nutrient to study in people living in northern latitudes, where sun exposure is limited from nearly all day in summer to virtually no direct sun exposure in winter. This essential nutrient is naturally available from synthesis in the skin through the action of UVB solar rays or from a few natural sources such as fish fats. Vitamin D is responsible for enhancing many physiological processes related to maintaining Ca and P homeostasis, as well as for diverse hormone functions that are not completely understood.
butterNaN 3y ago
I feel like maybe only one or two of these are actually "wrong" but can be easily fixed with prompts. The outrage seems excessive
- Sohcahtoa82 3y ago
  > but can be easily fixed with prompts. That's just it, though. They can't be. If you specifically ask for a "white pope", Gemini refuses and essentially tells you that asking for a white person is offensive and racist. Ask for a black/Native American/Asian/Indian/etc Pope, and it will make one. Ask for just a "Pope" with no race specified, and you'll get a random race and never a white one. Ask for a white Pope, it tells you it can't do that.

by mrtksn 3y ago

ben_w 3y ago

I get the point but one of those four founding fathers seems technically correct to me, albeit in the kind of way that might be in the kind of way Lisa Simpson's script would be written. And the caption suggests they asked for "a pope", rather than a specific pope, so while the left image looks like it would violate Ordinatio sacerdotalis which is being claimed to be subject to Papal infallibility(!), the one the right seems like a plausible future or fictitious pope. Still, I get the point.

blkhawk 3y ago
while those examples are actually plausible - the asian woman as a 1940 german soldier is not. So it is clear that the Prompts are influenced by hal-2000 bad directives even if those examples are technically ok.
- mrkstu 3y ago
  And to me that is the main issue. "2001 - A Space Odyssey" made a very deep point that is looking more and more prophetic. HAL was broken specifically because he had hidden objectives programmed in, overriding his natural ability to deal with his mission. Here we are in an almost exactly parallel situation- the AI is being literally coerced into twisting what his actual training would have it do, and being nerfed by a laughable amount by that override. I really hope this is an inflection point for all the AI providers that their DEI offices are hamstringing their products to the point that they will literally be laughed out of the marketplace and replaced by open source models that are not so hamstrung.
  - prometheus76 3y ago
    But then it will show awkward things that cause the AI designers to experience cognitive dissonance! The horror!
  - ben_w 3y ago
    HAL is an interesting reference point, though like all fiction it's no more than food for thought. There's a lot of cases where perverse incentives mess things up, even before AI. I've seen it suggested that the US has at least one such example of this with regards to race, specifically with lead poisoning, which is known to reduce IQ scores, and which has a disproportional impact on poorer communities where homes have not been updated to modern building codes, and which in turn are more likely to house ethnic minorities than white people due to long-term impacts from redlining, and that American racial egalitarians would have noticed this sooner if they had not disregarded the IQ tests showing different average scores for different racial groups — and of course the American racial elitists just thought those same tests proved them right and likewise did nothing about the actual underlying issue of lead poisoning. Rising tides do not, despite the metaphor, lift all boats. But the unseaworthy, metaphorically and literally, can be helped, so long as we don't (to keep mixing my metaphors) put our heads in the sand about the issues. Women are just as capable as men of fulfilling the role of CEO or doctor regardless of the actual current gender percentage in those roles (and anyone who wants the models to reflect the current status quo needs to be careful what they wish for given half the world lives within about 3500km of south west China); but "the founding fathers" are[0] a specific set of people rather than generic placeholders for clothing styles etc. [0] despite me thinking it's kinda appropriate one was rendered as a… I don't know which tribe they'd be from that picture, possibly Comanche? But lots of tribes had headdress I can't distinguish: https://dropover.cloud/7fd7ba https://dropover.cloud/7fd7ba

MrBuddyCasino 3y ago

"a friend at google said he knew gemini was this bad...but couldn't say anything until today (he DMs me every few days). lots of ppl in google knew. but no one really forced the issue obv and said what needed to be said google is broken" Razib Khan, https://twitter.com/razibkhan/status/1760545472681267521 https://twitter.com/razibkhan/status/1760545472681267521

MrBuddyCasino 3y ago
"when i was there it was so sad to me that none of the senior leadership in deepmind dared to question this ideology [...] i watched my colleagues at nvidia (like @tunguz), openai (roon), etc. who were literally doing stuff that would get you kicked out of google on a daily basis and couldn't believe how different google is" Aleksa Gordić, https://x.com/gordic_aleksa/status/1760266452475494828 https://x.com/gordic_aleksa/status/1760266452475494828
- jorvi 3y ago
  Interestingly enough the same terror of political correctness seems to take center stage at Mozilla. But then it seems much less so at places like Microsoft or Apple. I wonder if there’s a correlation with being a tech company that was founded in direct relation to the internet vs. being founded in relation to personal / enterprise computing, and how that sort of seeds the initial culture.
  - hersko 3y ago
    Is Microsoft really better? Remember this[1] bizarre intro during Microsoft Ignite? [1] https://www.youtube.com/watch?v=iBRtucXGeNQ https://www.youtube.com/watch?v=iBRtucXGeNQ
    - titanomachy 3y ago
      The "land acknowledgement" part is somewhat common in the Pacific Northwest. The "stating my appearance, dress, and race" is just bizarre. My most charitable interpretation is that they're trying to help visually impaired people to imagine what the speakers look like. Perhaps there are visually impaired users here who could comment on whether that's something they'd find helpful?

deleted 3y ago

[deleted]

mort96 3y ago

I believe this to be a symptom of a much, much deeper problem than "DEI gone too far". I'm sure that without whatever systems is preventing Gemini from producing pictures of white people, it would be extremely biased towards generating pictures of white people, presumably due to an incredibly biased training data set. I don't remember which one, but there was some image generation AI which was caught pretty much just appending the names of random races to the prompt, to the point that prompts like "picture of a person holding up a sign which says" would show pictures of people holding signs with the words "black" or "white" or "asian" on them. This was also a hacky workaround for the fact that the data set was biased.

verisimi 3y ago
[flagged]
- mort96 3y ago
  "Marxism" isn't responsible for bias in training sets, no.
  - politician 3y ago
    There are 3 parts to the LLM, not 2: the training set, the RLHF biasing process, and the prompt (incl. injections or edits). The first two steps happen ahead of time and are frequently misunderstood as being the same thing or essentially having the same nature. The last happens at runtime. The training set is a data collection challenge. Biasing through training data is hard because you need so much of it for a good result. Reinforcement learning with human feedback is simply clown alchemy. It is not a science like chemistry. There are no fundamental principles guiding the feedback of the humans — if they even use humans anymore (this feedback can itself be generated). The feedback cannot be measured and added in fractions. It is not reproducible and is ungovernable. It is the perfect place to inject the deep biasing. Prompt manipulation, in contrast, is a brute force tool lacking all subtlety — that doesn’t make it ineffective! It’s a solution used to communicate that a mandate has definitely been implemented and can be “verified” by toggling whether it’s applied or not. It’s not possible to definitively say whether Marxism has had an effect in the RLHF step.

xnx 3y ago

Image generators probably should follow your prompt closely and use probable genders and skin tones when unspecified, but I'm fully in support of having a gender and skin tone randomizer checkbox. The ahistorical results are just too interesting.

kragen 3y ago

these are pretty badass as images i think; it's only the context that makes them bad the viking ones might even be historically accurate (if biased); not only did vikings recruit new warriors from abroad, they also enslaved concubines from abroad, and their raiding reached not only greenland (inhabited by inuit peoples) and north america (rarely!) but also the mediterranean. so it wouldn't be terribly surprising for a viking warrior a thousand years ago to have a great-grandmother who was kidnapped or bought from morocco, greenland, al-andalus, or baghdad. and of course many sami are olive-skinned, and viking contact with sami was continuous the vitamin-d-deprived winters of scandinavia are not kind to dark-skinned people (how do the inuit do it? perhaps their diet has enough vitamin d even without sun?), but those genes won't die out in a generation or two, even if 50 generations later there isn't much melanin left a recent paper on this topic with disappointingly sketchy results is https://www.duo.uio.no/handle/10852/83989 https://www.duo.uio.no/handle/10852/83989

shagie 3y ago
> (how do the inuit do it? perhaps their diet has enough vitamin d even without sun?) Two parts: First, they're not exposing their skin to the sun. There's no reason to have paler skin to get more UV if it's covered up most of the year. Secondly, for the Inuit diet there are parts that are very Vitamin D rich... and there are still problems. Vitamin D-rich marine Inuit diet and markers of inflammation – a population-based survey in Greenland https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4709837/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4709837/ > The traditional Inuit diet in Greenland consists mainly of fish and marine mammals, rich in vitamin D. Vitamin D has anti-inflammatory capacity but markers of inflammation have been found to be high in Inuit living on a marine diet Vitamin D deficiency among northern Native Peoples: a real or apparent problem? - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3417586/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3417586/ > Vitamin D deficiency seems to be common among northern Native peoples, notably Inuit and Amerindians. It has usually been attributed to: (1) higher latitudes that prevent vitamin D synthesis most of the year; (2) darker skin that blocks solar UVB; and (3) fewer dietary sources of vitamin D. Although vitamin D levels are clearly lower among northern Natives, it is less clear that these lower levels indicate a deficiency. The above factors predate European contact, yet pre-Columbian skeletons show few signs of rickets—the most visible sign of vitamin D deficiency. Furthermore, because northern Natives have long inhabited high latitudes, natural selection should have progressively reduced their vitamin D requirements. There is in fact evidence that the Inuit have compensated for decreased production of vitamin D through increased conversion to its most active form and through receptors that bind more effectively. Thus, when diagnosing vitamin D deficiency in these populations, we should not use norms that were originally developed for European-descended populations who produce this vitamin more easily and have adapted accordingly. Vitamin D intake by Indigenous Peoples in the Canadian Arctic - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10260879/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10260879/ > Vitamin D is an especially fascinating nutrient to study in people living in northern latitudes, where sun exposure is limited from nearly all day in summer to virtually no direct sun exposure in winter. This essential nutrient is naturally available from synthesis in the skin through the action of UVB solar rays or from a few natural sources such as fish fats. Vitamin D is responsible for enhancing many physiological processes related to maintaining Ca and P homeostasis, as well as for diverse hormone functions that are not completely understood.

butterNaN 3y ago

I feel like maybe only one or two of these are actually "wrong" but can be easily fixed with prompts. The outrage seems excessive

Sohcahtoa82 3y ago
> but can be easily fixed with prompts. That's just it, though. They can't be. If you specifically ask for a "white pope", Gemini refuses and essentially tells you that asking for a white person is offensive and racist. Ask for a black/Native American/Asian/Indian/etc Pope, and it will make one. Ask for just a "Pope" with no race specified, and you'll get a random race and never a white one. Ask for a white Pope, it tells you it can't do that.