6 ms·

> Reducing bias: We implemented a new technique so that DALL·E generates images of people that more accurately reflect the diversity of the world’s population.

> Reducing bias: We implemented a new technique so that DALL·E generates images of people that more accurately reflect the diversity of the world’s population. This technique is applied at the system level when DALL·E is given a prompt about an individual that does not specify race or gender, like “CEO.” Will it do it "more accurately" as they claim? As in, if 90% of CEOs are male, then the odds of a CEO being male in a picture is 90%? Or less "accurately reflect the diversity of the world’s population" and show what they would like the real world to be like?

scifibestfi 4y ago
The latter. Here's what we, a small number of people, think the world should look like according to our own biases and information bubble in the current moment. We will impose our biases upon you, the unenlightened masses who must be manipulated for your own good. And for god sakes, don't look for photos of the US Math team or NBA Basketball or compare soccer teams across different countries and cultures.
- bequanna 4y ago
  > Here's what we, a small number of people, think the world should look like according to our own biases and information bubble in the current moment. You're being quite charitable. It is much more likely that optics and virtue signaling is behind this addition.
  - erikpukinskis 4y ago
    If I search for “food” I don’t want to see a slice of pizza every time, even if that’s the #1 food. I want to see some variety. I think you’re jumping to quickly to bad intentions. Injecting diversity of results is a sane thing to do, totally irrespective of politics.
    - jlawson 4y ago
      You are correct but that's not what anyone's discussing. If I search for "food", the reasonable result would be to get images that represent food according to its actual proportions of real life. E.g. if Pizza is the most common food at 10% prevalence, 10% of the images should be pizza. That's not what OpenAI are doing. They are introducing crafted biases to create images that deliberately misrepresent what the world looks like, and instead represent what they believe the world ought to look like. -- You also need some reason why diversity "of this" is important but not diversity "of that". Why is diversity of race and sex so critical, but not diversity of age, height, disability? Should a search for "basketball player" yield 1/2 able-bodied people and 1/2 wheelchair basketball players? Why? Then try to answer where you came up with the categories you do want depicted. Why are the races what they are? Should "basketball player" include half whites and half black people? Or maybe split in 3, white/black/Asian? Why not Australian Aborigines, native Americans, or Persians - so we can divide into 6? If you don't add Indian people to your list then, is that racist against them? How did you decide what must be represented, in what proportions, and what's okay to leave out?
klohto 4y ago
hardmaru on Twitter has examples. It’s the second, the one they would like it to be.
kache_ 4y ago
They literally just add "black" and "female" with some weight before any prompt containing person. A comical work around to so called "bias" (isn't the whole point of these models to encode some bias?). Here's some experimentation showing this. https://twitter.com/rzhang88/status/1549472829304741888 https://twitter.com/rzhang88/status/1549472829304741888 As competitors with lower price points prop up, you'll see everyone ditch models with "anti bias" measures and take their $ somewhere else. Or maybe we'll get some real solution, that adds noise to the embeddings, and not some half assed workaround to the arbitrary rules that your resident AI Ethicist comes up with.
- danielvf 4y ago
  Add after. So you can see the added words by making a prompt like "a person holding a sign saying ", and then the sign says the extra words if they are added.
  - kache_ 4y ago
    Yeah actually, good call. The position of the token matters, since these things use transformers to encode the embeddings. https://www.assemblyai.com/blog/how-imagen-actually-works/ https://www.assemblyai.com/blog/how-imagen-actually-works/
- whywhywhywhy 4y ago
  How does it deal with bias that is negative? Would only work for positive biases where if they actually want to equalize it then it needs to be adding the opposite to negative biases. To counteract the bias of their dataset they need to have someone sitting there actively thinking in bias to counteract the bias with anti-bias seasoning for every bias causing term. Feel bad for whatever person is tasked with that job. Could always just fix your dataset, but who's got time and money to do that /s
totetsu 4y ago
Yes the quality of surrealist generations went down with that change suddenly including gender and race into prompts that I really didn't want anything specific in. Like a snail radio DJ, and suddenly the microphone is a woman of colours head.. I understand the intention but I want this to be a default on but you can turn it off thing.
TheFreim 4y ago
It's also odd since you'd think that this would be an issue solved by training with representative images in the first place. If you used good input you'd expect an appropriate output, I don't know why manual intervention would be necessary unless it's for other purposes than stated. I suspect this is another case where "diversity" simply means "less whites".
bjt2n3904 4y ago
Will it reduce bias across all fields? Or only ones that are desirable? How about historical? "A photo of a group of soldiers from WW2 celebrating victory over nazi CEOs and plumbers".
naillo 4y ago
It's also funny that this likely won't 'unbias' any actual published images coming out of it. If 90% of the images in the world has a male CEO, then for whatever reason that's the image people will pick and choose from DALL-Es output. (Generalized to any unbiasing - i.e. they'll be debiased by humans.)
- bequanna 4y ago
  Imagine you're in South Korea (or any other ethnically homogenous country). Do you want "black" "female" randomly appended to your input?
  - educaysean 4y ago
    If I was using this in South Korea, how is showing all white people any better than showing whites, blacks, latinos and asians?
    - bequanna 4y ago
      You would presumably input “South Korean CEO”. DALL-E would then unhelpfully add “black” “female” without your knowledge.
      - educaysean 4y ago
        I just tried it out and it looks like DALL-E isn't as inept as you imagined. Exact query used was 'A profile photo of a male south korean CEO', and it spat out 4 very believable korean business dudes. Supplying the race and sex information seems to prevent new keywords from being injected. I see no problem with the system generating female CEOs when the gender information is omitted, unless you think there are?
        bequanna 4y ago
        Isn’t the diversity keyword injection random? My point is that it is pointless. If you want an image of a <race> <gender> person included, you can just specify it yourself.
StrictDabbler 4y ago
If accurately reflects the world population then only one in six pictures will be a white person. Half the pictures will be Asian, another sixth will be Indian. Slightly more than half of the pictures will be women. That accurately represents the world's diversity. It won't accurately reflect the world's power balance but that doesn't seem to be their goal. If you want to say "white male CEO" because you want results that support the existing paradigm it doesn't sound like they'll stop you. I can't imagine a more boring request. Let's look at interesting questions: If you ask for "victorian detective" are you going to get a bunch of Asians in deerstalker caps with pipes? What about Jedi? A lot of the Jedi are blue and almost nobody on Earth is. Are cartoon characters exempt from the racial algorithm? If I ask for a Smurf surfing on a pizza I don't think that making the Smurf Asian is going to be a comfortable image for any viewer. What about ageism? 16% of the population is over sixty. Will a request for "superhero lifting a building" have an 16% chance of being old? If I request a "bad driver peering over a steering wheel" am I still going to get an Asian 50% of the time? Are we ok with that? I respect the team's effort to create an inclusive and inoffensive tool. I expect it's going to be hard going.
- bequanna 4y ago
  > inoffensive tool. Wouldn't that result end up being like "inoffensive art" or "inoffensive comedy"? Bland, boring and Corporate-PC.
  - driverdan 4y ago
    To a certain degree, yes. They care more about the image of the project than art. Considering a large amount of art depicts non-sexual nudity yet they block all nudity, art is not their primary concern.
    - bequanna 4y ago
      Some people claim to be emotionally "triggered" by images of police. Does that mean DALL-E should also start blocking images that contain police?
  - 4y ago
president 4y ago
Most likely this was something forced by their marketing team or their office of diversity. Given the explanation of the implementation (arbitrarily adding "black" and "female" qualifiers), it's clear it was just an afterthought.
deleted 4y ago
[deleted]
noelsusman 4y ago
In their examples, the "After mitigation" photos seem more representative of the real world. Before you got nothing but white guys for firefighter or software engineer and nothing but white ladies for teacher. That's not how the real world actually is today. I'm not sure how they would accomplish 100% accurate proportions anyway, or even why that would be desirable. If I don't specify any traits then I want to see a wide variety of people. That's a more useful product than one that just gives me one type of person over and over again because it thinks there are no female firefighters in the world.

scifibestfi 4y ago

The latter. Here's what we, a small number of people, think the world should look like according to our own biases and information bubble in the current moment. We will impose our biases upon you, the unenlightened masses who must be manipulated for your own good. And for god sakes, don't look for photos of the US Math team or NBA Basketball or compare soccer teams across different countries and cultures.

bequanna 4y ago
> Here's what we, a small number of people, think the world should look like according to our own biases and information bubble in the current moment. You're being quite charitable. It is much more likely that optics and virtue signaling is behind this addition.
- erikpukinskis 4y ago
  If I search for “food” I don’t want to see a slice of pizza every time, even if that’s the #1 food. I want to see some variety. I think you’re jumping to quickly to bad intentions. Injecting diversity of results is a sane thing to do, totally irrespective of politics.
  - jlawson 4y ago
    You are correct but that's not what anyone's discussing. If I search for "food", the reasonable result would be to get images that represent food according to its actual proportions of real life. E.g. if Pizza is the most common food at 10% prevalence, 10% of the images should be pizza. That's not what OpenAI are doing. They are introducing crafted biases to create images that deliberately misrepresent what the world looks like, and instead represent what they believe the world ought to look like. -- You also need some reason why diversity "of this" is important but not diversity "of that". Why is diversity of race and sex so critical, but not diversity of age, height, disability? Should a search for "basketball player" yield 1/2 able-bodied people and 1/2 wheelchair basketball players? Why? Then try to answer where you came up with the categories you do want depicted. Why are the races what they are? Should "basketball player" include half whites and half black people? Or maybe split in 3, white/black/Asian? Why not Australian Aborigines, native Americans, or Persians - so we can divide into 6? If you don't add Indian people to your list then, is that racist against them? How did you decide what must be represented, in what proportions, and what's okay to leave out?

klohto 4y ago

hardmaru on Twitter has examples. It’s the second, the one they would like it to be.

kache_ 4y ago

They literally just add "black" and "female" with some weight before any prompt containing person. A comical work around to so called "bias" (isn't the whole point of these models to encode some bias?). Here's some experimentation showing this. https://twitter.com/rzhang88/status/1549472829304741888 https://twitter.com/rzhang88/status/1549472829304741888 As competitors with lower price points prop up, you'll see everyone ditch models with "anti bias" measures and take their $ somewhere else. Or maybe we'll get some real solution, that adds noise to the embeddings, and not some half assed workaround to the arbitrary rules that your resident AI Ethicist comes up with.

danielvf 4y ago
Add after. So you can see the added words by making a prompt like "a person holding a sign saying ", and then the sign says the extra words if they are added.
- kache_ 4y ago
  Yeah actually, good call. The position of the token matters, since these things use transformers to encode the embeddings. https://www.assemblyai.com/blog/how-imagen-actually-works/ https://www.assemblyai.com/blog/how-imagen-actually-works/
whywhywhywhy 4y ago
How does it deal with bias that is negative? Would only work for positive biases where if they actually want to equalize it then it needs to be adding the opposite to negative biases. To counteract the bias of their dataset they need to have someone sitting there actively thinking in bias to counteract the bias with anti-bias seasoning for every bias causing term. Feel bad for whatever person is tasked with that job. Could always just fix your dataset, but who's got time and money to do that /s

totetsu 4y ago

Yes the quality of surrealist generations went down with that change suddenly including gender and race into prompts that I really didn't want anything specific in. Like a snail radio DJ, and suddenly the microphone is a woman of colours head.. I understand the intention but I want this to be a default on but you can turn it off thing.

TheFreim 4y ago

It's also odd since you'd think that this would be an issue solved by training with representative images in the first place. If you used good input you'd expect an appropriate output, I don't know why manual intervention would be necessary unless it's for other purposes than stated. I suspect this is another case where "diversity" simply means "less whites".

bjt2n3904 4y ago

Will it reduce bias across all fields? Or only ones that are desirable? How about historical? "A photo of a group of soldiers from WW2 celebrating victory over nazi CEOs and plumbers".

naillo 4y ago

It's also funny that this likely won't 'unbias' any actual published images coming out of it. If 90% of the images in the world has a male CEO, then for whatever reason that's the image people will pick and choose from DALL-Es output. (Generalized to any unbiasing - i.e. they'll be debiased by humans.)

bequanna 4y ago
Imagine you're in South Korea (or any other ethnically homogenous country). Do you want "black" "female" randomly appended to your input?
- educaysean 4y ago
  If I was using this in South Korea, how is showing all white people any better than showing whites, blacks, latinos and asians?
  - bequanna 4y ago
    You would presumably input “South Korean CEO”. DALL-E would then unhelpfully add “black” “female” without your knowledge.
    - educaysean 4y ago
      I just tried it out and it looks like DALL-E isn't as inept as you imagined. Exact query used was 'A profile photo of a male south korean CEO', and it spat out 4 very believable korean business dudes. Supplying the race and sex information seems to prevent new keywords from being injected. I see no problem with the system generating female CEOs when the gender information is omitted, unless you think there are?
      - bequanna 4y ago
        Isn’t the diversity keyword injection random? My point is that it is pointless. If you want an image of a <race> <gender> person included, you can just specify it yourself.

StrictDabbler 4y ago

If accurately reflects the world population then only one in six pictures will be a white person. Half the pictures will be Asian, another sixth will be Indian. Slightly more than half of the pictures will be women. That accurately represents the world's diversity. It won't accurately reflect the world's power balance but that doesn't seem to be their goal. If you want to say "white male CEO" because you want results that support the existing paradigm it doesn't sound like they'll stop you. I can't imagine a more boring request. Let's look at interesting questions: If you ask for "victorian detective" are you going to get a bunch of Asians in deerstalker caps with pipes? What about Jedi? A lot of the Jedi are blue and almost nobody on Earth is. Are cartoon characters exempt from the racial algorithm? If I ask for a Smurf surfing on a pizza I don't think that making the Smurf Asian is going to be a comfortable image for any viewer. What about ageism? 16% of the population is over sixty. Will a request for "superhero lifting a building" have an 16% chance of being old? If I request a "bad driver peering over a steering wheel" am I still going to get an Asian 50% of the time? Are we ok with that? I respect the team's effort to create an inclusive and inoffensive tool. I expect it's going to be hard going.

bequanna 4y ago
> inoffensive tool. Wouldn't that result end up being like "inoffensive art" or "inoffensive comedy"? Bland, boring and Corporate-PC.
- driverdan 4y ago
  To a certain degree, yes. They care more about the image of the project than art. Considering a large amount of art depicts non-sexual nudity yet they block all nudity, art is not their primary concern.
  - bequanna 4y ago
    Some people claim to be emotionally "triggered" by images of police. Does that mean DALL-E should also start blocking images that contain police?
- 4y ago

president 4y ago

Most likely this was something forced by their marketing team or their office of diversity. Given the explanation of the implementation (arbitrarily adding "black" and "female" qualifiers), it's clear it was just an afterthought.

deleted 4y ago

[deleted]

noelsusman 4y ago

In their examples, the "After mitigation" photos seem more representative of the real world. Before you got nothing but white guys for firefighter or software engineer and nothing but white ladies for teacher. That's not how the real world actually is today. I'm not sure how they would accomplish 100% accurate proportions anyway, or even why that would be desirable. If I don't specify any traits then I want to see a wide variety of people. That's a more useful product than one that just gives me one type of person over and over again because it thinks there are no female firefighters in the world.