12 ms·
Here's the problem for Google: Gemini pukes out a perfect visual representation of actual systemic racism that pervades throughout modern corporate culture in t
by 34679 3y ago
Here's the problem for Google: Gemini pukes out a perfect visual representation of actual systemic racism that pervades throughout modern corporate culture in the US. Daily interactions can be masked by platitudes and dog whistles. A poster of non-white celtic warriors cannot.
Gemini refused to create an image of "a nice white man", saying it was "too spicy", but had no problem when asked for an image of "a nice black man".
- octacat 3y agosounds too close to "nice guy", that is why "spicy". Nice guys finish last... Yea, people broke "nice" word in general.
- wakawaka28 3y agoIt's so stubborn it generated pictures of diverse Nazis, and that's what I saw a liberal rag leading with. In fact it is almost impossible to get a picture of a white person out of it.
- ActionHank 3y agoI love the idea of testing an image gen to see if it generates multicultural ww2 nazis because it is just so contradictory.
- aniftythrifrty 3y agoOf course it's not that different from today.
- Perceval 3y agoThe media have done a number of stories, analyses, and editorials noting that white nationalist/white supremacy groups appear to be more diverse than one would otherwise expect: * https://www.brennancenter.org/our-work/research-reports/why-white-supremacist-groups-attract-latinos-their-ranks https://www.brennancenter.org/our-work/research-reports/why-... * https://www.washingtonpost.com/national-security/minorities-far-right-visible-role/2021/05/16/e7ba8338-a915-11eb-8c1a-56f0cb4ff3b5_story.html https://www.washingtonpost.com/national-security/minorities-... * https://www.aljazeera.com/opinions/2023/6/2/why-white-supremacists-are-not-always-white https://www.aljazeera.com/opinions/2023/6/2/why-white-suprem... * https://www.voanews.com/a/why-some-nonwhite-americans-espouse-right-wing-extremism-/7101229.html https://www.voanews.com/a/why-some-nonwhite-americans-espous... * https://www.washingtonpost.com/politics/2023/05/08/texas-shooting-white-supremacy-non-whites/ https://www.washingtonpost.com/politics/2023/05/08/texas-sho... * https://www.latimes.com/california/story/2021-08-20/recall-candidate-larry-elder-is-a-threat-to-black-californians https://www.latimes.com/california/story/2021-08-20/recall-c...
- BlueTemplar 3y agoThis "What if Civilization had lyrics ?" skit comes to mind : https://youtube.com/watch?v=aL6wlTDPiPU https://youtube.com/watch?v=aL6wlTDPiPU
- vidarh 3y agoAnd as someone far to the left of a US style "liberal", that is equally offensive and racist as only generating white people. Injecting fake diversity into situations where it is historically inaccurate is just as big a problem as erasing diversity where it exists. The Nazi example is stark, and perhaps too stark, in that spreading fake notions of what they look like seems ridiculous now, but there are more borderline examples where creating the notion that there was more equality than there really was, for example, downplays systematic historical inequities. I think you'll struggle to find people who want this kind of "diversity*. I certainly don't. Getting something representative matters, but it also needs to reflect reality.
- CoastalCoder 3y agoI think you hit on an another important issue: Do people want the generated images to be representative, or aspirational?
- vidarh 3y agoI think there's a large overlap there, in that in media, to ensure an experience of representation you often need to exaggerate minority presence (and not just in terms of ethnicity or gender) to create a reasonable impression, because if you "round down" you'll often end up with a homogeneous mass that creates impressions of bias in the other direction. In that sense, it will often end up aspirational. E.g. let's say you're making something about a population with 5% black people, and you're presenting a group of 8. You could justify making that group entirely white very easily - you've just rounded down, and plenty of groups of 8 within a population like that will be all white (and some will be all black). But you're presenting a narrow slice of an experience of that society, and not including a single black person without reason makes it easy to create an impression of that population as entirely white. But it also needs to at scale be representative within plausible limits, or it just gets insultingly dumb or even outright racist, just against a different set of people.
- Scea91 3y agoI think you can be aspirational for the future but I can't see how a request for an image in historical context can ever be desired to be aspirational instead of realistic? On a second thought, maybe for requests like "picture of a crowd cheering signing of the declaration of independence" the exists a big public demand for images that are more diverse than reality was? However, there are many reasons to prefer historical accuracy even here.
- scarface_74 3y agoChatGPT won’t draw a picture of a “WW2 German soldier riding a horse”. Makes sense. But it won’t even draw “a picture of a modern German soldier riding a horse”. Are Germans going to be tarnished forever? FWIW: I’m a black guy not an undercover Nazi sympathizer. But I do want my computer to do what I tell it to do.
- mike_hearn 3y agoTried and it refused on the basis of having military themes, and explained it was due to content restrictions. Restricting images of war seems kind of silly given the prevalence of hyper-realistic video games that simulate war in gory detail, but it's not related to the reasons for Gemini going wrong.
- scarface_74 3y agoHowever, ChatGPT 4 would draw a picture of an American soldier during WW2 riding a horse.
- mike_hearn 3y agoYou're right. That refusal is disappointing.
- deleted 3y ago[deleted]
- empath-nirvana 3y agoThere is an _actual problem_ that needs to be solved. If you ask generative AI for a picture of a "nurse", it will produce a picture of a white woman 100% of the time, without some additional prompting or fine tuning that encourages it to do something else. If you ask a generative AI for a picture of a "software engineer", it will produce a picture of a white guy 100% of the time, without some additional prompting or fine tuning that encourages it to do something else. I think most people agree that this isn't the optimal outcome, even assuming that it's just because most nurses are women and most software engineers are white guys, that doesn't mean that it should be the only thing it ever produces, because that also wouldn't reflect reality -- there are lots of non white male software developers. There is a couple of difficulties in solving this. If you ask it to be "diverse" and ask it to generate _one person_, it's going to almost always pick the non-white non-male option (again because of societal biases about what 'diversity' means), so you probably have to have some cleverness in prompt injection to get it to vary its outcome. And then you also need to account for every case where "diversity" as defined in modern America is actually not an accurate representation of a population. In particular, the racial and ethnic makeup of different countries are often completely different from each other, some groups are not-diverse in fact and by design, and historically, even within the same country, the racial and ethnic makeup of countries has changed over time. I am not sure it's possible to solve this problem without allowing the user to control it, and to try and do some LLM pre-processing to determine if and whether diversity is appropriate to the setting as a default.
- Jensson 3y agoDiversity isn't just a default here, it does it even when explicitly asked for a specific outcome. Diversity as a default wouldn't be a big deal, just ask for what you want, forced diversity however is a big a problem since it means you simply can't generate many kind of images.
- D13Fd 3y ago> If you ask generative AI for a picture of a "nurse", it will produce a picture of a white woman 100% of the time, without some additional prompting or fine tuning that encourages it to do something else. > If you ask a generative AI for a picture of a "software engineer", it will produce a picture of a white guy 100% of the time, without some additional prompting or fine tuning that encourages it to do something else. What should the result be? Should it accurately reflect the training data (including our biases)? Should we force the AI to return results in proportion to a particular race/ethnicity/gender's actual representation in the workplace? Or should it return results in proportion to their representation in the population? But the population of what country? The results for Japan or China are going to be a lot different than the results for the US or Mexico, for example. Every country is different. I'm not saying the current situation is good or optimal. But it's not obvious what the right result should be.
- ajross 3y ago> actual systemic racism that pervades throughout modern corporate culture Ooph. The projection here is just too much. People jumping straight across all the reasonable interpretations straight to the maximal conspiracy theory. Surely this is just a bug. ML has always had trouble with "racism" accusations, but for years it went in the other direction. Remember all the coverage of "I asked for a picture of a criminal and it would only give me a black man", "I asked it to write a program the guess the race of a poor person and it just returned 'black'", etc... It was everywhere. So they put in a bunch of upstream prompting to try to get it to be diverse. And clearly they messed it up. But that's not "systemic racism", it's just CYA logic that went astray.
- seanw444 3y agoI mean, the model would be making wise guesses based on the statistics.
- ajross 3y agoOooph again. Which is the root of the problem. The statement "All American criminals are black" is, OK, maybe true to first order (I don't have stats and I'm not going to look for them). But, first, on a technical level first order logic like that leads to bad decisions. And second, it's clearly racist. And people don't want their products being racist. That desire is pretty clear, right? It's not "systemic racism" to want that, right?
- itsoktocry 3y ago>"All American criminals are black" I'm not even sure it's worth arguing, but who ever says that? Why go to a strawman? However, looking at the data, if you see that X race commits crime (or is the victim of crime) at a rate disproportionate to their place in the population, is that racist? Or is it useful to know to work on reducing crime?
- ajross 3y ago> I'm not even sure it's worth arguing, but who ever says that? Why go to a strawman? The grandparent post called a putative ML that guessed that all criminals were black a "wise guess", I think you just missed the context in all the culture war flaming?
- jelling 3y agoI had the same problem while designing an AI related tool and the solution is simple: ask the user a clarifying question as to whether they want a specific ethnic background or default to random. No matter what technical solution they come up with, even if there were one, it will be a PR disaster. But if they just make the user choose the problem is solved.
- rondini 3y agoAre you seriously claiming that the actual systemic racism in our society is discrimination against white people? I just struggle to imagine someone holding this belief in good faith.
- vdaea 3y agoHe didn't say "our society", he said "modern corporate culture in the US"
- dylan604 3y agoLuckily for you, you don't have to imagine it. There are groups of people that absolutely believe that modern society has become anti-white. Unfortunately, they have found a megaphone with internet/social platforms. However, just because someone believes something doesn't make it true. Take flat Earthers as a less hate filled example.
- klyrs 3y agoThat take is extremely popular on HN
- silent_cal 3y agoI think it's obviously one of the problems.
- itsoktocry 3y ago>I just struggle to imagine someone holding this belief in good faith. Because you're racist against white people. "All white people are privileged" is a racist belief.
- wandddt 3y agoSaying "being white in the US is a privilege" is not the same thing as saying "all white people have a net positive privilege". The former is accurate, the latter is not. Usually people mean the former, even if it's not explicitly said.
- 3y ago
- silent_cal 3y agoI thought you were going to say anti-white racism.
- giraffe_lady 3y agoI think they did? It's definitely unclear but after looking at it for a minute I do read it as referring to racism against white people.
- silent_cal 3y agoI thought he was saying that diversity efforts like this are "platitudes" and not really addressing the root problems. But also not sure.
- stronglikedan 3y ago> actual systemic racism That's a bogeyman. There's racism for sure, especially since 44 greatly rejuvenated it during his term, but it's far from systematic.
- 2OEH8eoCRo0 3y agoDEI isn't systemic? It's racism as part of a system.
- deleted 3y ago[deleted]
- commandlinefan 3y agoEverybody seems to be focusing on the actual outcome while ignoring the more disconcerting meta-problem: how in the world _could_ an AI have been trained that would produce a black Albert Einstein? What was it even trained _on_? This couldn't have been an accident, the developers had to have bent over backwards to make this happen, in a really strange way.
- lolinder 3y agoThis isn't very surprising if you've interacted much with these models. Contrary to the claims in the various lawsuits, they're not just regurgitating images they've seen before, they have a good sense of abstract concepts and can pretty easily combine ideas to make things that have never been seen before. This type of behavior has been evident ever since DALL-E's horse-riding astronaut [0]. There's no training image that resembles it (the astronaut even has their hands in the right position... mostly), it's combining ideas about what a figure riding a horse looks like and what an astronaut looks like. Changing Albert Einstein's skin color should be even easier. [0] https://www.technologyreview.com/2022/04/06/1049061/dalle-openai-gpt3-ai-agi-multimodal-image-generation/ https://www.technologyreview.com/2022/04/06/1049061/dalle-op...
- mpweiher 3y ago> Contrary to the claims in the various lawsuits, they're not just regurgitating images they've seen before, I don't think "just" is what the lawsuits are saying. It's the fact that they can regurgitate a larger subset (all?) of the original training data verbatim. At some point, that means you are copying the input data, regardless of how convoluted the tech underneath.
- lolinder 3y agoFair, I should have said something along the lines of "contrary to popular conception of the lawsuits". I haven't actually followed the court documents at all, so I was actually thinking of discussions in mainstream and social media.
- TacticalCoder 3y ago> A poster of non-white celtics warriors cannot > refused to create an image of 'a nice white man' This is anti-white racism. Plain and simple. It's insane to see how some here are playing with words to try to explain how this is not what it is. It is anti-white racism and you are playing with fire if you refuse to acknowledge it. My family is of all the colors: white, yellow and black. Nieces and nephews are more diverse than woke people could dream of... And we reject and we ll fight this very clear anti-white racism.
- shitpostbot 3y agoSeems like all they need to do is, when prompted to generate images of people, ask for clarification on whether the user wants to constrain the appearance, use the default output of the model, or otherwise offer to modify the prompt to reduce biases or however they would describe it. Doesn't even have to be interactive, just a note on the side would be enough. Ultimately the only "real" concern was silently perpetuating biases, as long as it isn't silent and the user is made aware of the options, who cares? You'll never be able to baby-proof these things enough to stop "bad actors" from generating whatever they want without compromising the actual usage
- Mean-Greek 3y ago[flagged]