5 ms·

Translation: we need to hand-tune this to not reflect reality but instead the world as we (Caucasian/Asian male American woke upper-middle class San Fransisco e

Translation: we need to hand-tune this to not reflect reality but instead the world as we (Caucasian/Asian male American woke upper-middle class San Fransisco engineers) wish it to be. Maybe that's a nice thing, I wouldn't say their values are wrong but let's call a spade a spade.

ceejayoz 4y ago
"Reality" as defined by the available training set isn't necessarily reality. For example, Google's image search results pre-tweaking had some interesting thoughts on what constitutes a professional hairstyle, and that searches for "men" and "women" should only return light-skinned people: https://www.theguardian.com/technology/2016/apr/08/does-google-unprofessional-hair-results-prove-algorithms-racist- https://www.theguardian.com/technology/2016/apr/08/does-goog... Does that reflect reality? No. (I suspect there are also mostly unstated but very real concerns about these being used as child pornography, revenge porn, "show my ex brutally murdered" etc. generators.)
- ceeplusplus 4y ago
  The reality is that hair styles on the left side of the image in the article are widely considered unprofessional in today's workplaces. That may seem egregiously wrong to you, but it is a truth of American and European society today. Should it be Google's job to rewrite reality?
  - rcMgD2BwE72F 4y ago
    In any case, Google will be writing their reality. Who picked the image sample for the ML to run on, if not Google? What's the problem with writing it again, then? They know their biases and want to act on it. It's like blaming a friend for trying to phrase things nicely, and telling them to speak headlong with zero concern for others instead. Unless you believe anyone trying to do good is being hypocrite… I, for one, like civility.
  - ceejayoz 4y ago
    The "unprofessional" results are almost exclusively black women; the "professional" ones are almost exclusively white or light skinned. Unless you think white women are immune to unprofessional hairstyles, and black women incapable of them, there's a race problem illustrated here even if you think the hairstyles illustrated are fairly categorized.
    - rvnx 4y ago
      If you type as a prompt "most beautiful woman in the world", you get a brown-skinned brown-haired woman with hazel eyes. What should be the right answer then ? You put a blonde, you offend the brown haired. You put blue eyes, you offend the brown eyes. etc.
      - ceejayoz 4y ago
        That's an unanswerable question. Perhaps the answer is "don't". Siri takes this approach for a wide range of queries.
        nomel 4y ago
        How do you pick what should and shouldn't be restricted? Is there some "offense threshold"? I suspect all queries relating to religion, ethnicity, sexuality, and gender will need to be restricted, which almost certainly means you probably can't include humans at all, other than ones artificially inserted with mathematically proven random attributes. Maybe that's why none are in this demo.
        daenz 4y ago
        "Is Taiwan a country" also comes to mind.
        rvnx 4y ago
        What would a human who can freely speak without morale or being judged say on average after having ingested all the information on the internet ?
  - deleted 4y ago
    [deleted]
  - colinmhayes 4y ago
    Only black people have unprofessional hair and only white people have professional hair is not reality.
- rvnx 4y ago
  If your query was about hairstyle, why do you even look or care about the skin color ? Nowhere there is any precision for a preferred skin color in the query of th user. So it sorts and gives the most average examples based on the examples that were found on the internet. Essentially answering the query "SELECT * FROM `non-professional hairstyles` ORDER BY score DESC LIMIT 10". It's like if you search on Google "best place for wedding night". You may get 3 places out of 10 in Santorini, Greece. Yes you could have an human remove these biases because you feel that Sri Lanka is the best place for a wedding, but what if there is a consensus that Santorini is really the most appraised in the forums or websites that were crawled by Google ?
  - ceejayoz 4y ago
    > The algorithm is just ranking the top "non-professional hairstyle" in the most neutral way in its database You're telling me those are all the most non-professional hairstyles available? That this is a reasonable assessment? That fairly standard, well-kept, work-appropriate curly black hair is roughly equivalent to the pink-haired, three-foot-wide hairstyle that's one of the only white people in the "unprofessional" search? Each and everyone of them is less workplace appropriate than, say, http://www.7thavenuecostumes.com/pictures/750x950/P_CC_70594.jpg http://www.7thavenuecostumes.com/pictures/750x950/P_CC_70594... ?
    - rvnx 4y ago
      I'm saying that the dataset needs to be expanded to cover the most examples possible. Work a lot on adding even more examples, in order to make the algorithms as close as possible to the "average reality". At some point we may even ultimately reach the state that the robots even collect intelligence directly in the real world, and not on the internet (even closer to reality). Censoring results sounds the best recipe for a dystopian world where only one view is right.
  - jayd16 4y ago
    The results are not inherently neutral because the database is from non-neutral input. It's a simple case of sample bias.
- userbinator 4y ago
  unstated but very real concerns I say let people generate their own reality. The sooner the masses realise that ceci n'est pas une pipe , the less likely they are to be swayed by the growing un-reality created by companies like Google.
- ChadNauseam 4y ago
  You know, it wouldn't surprise me if people talking about how black curly hair shouldn't be seen as unprofessional contributed to google thinking there's an association between the concepts of "unprofessional hair" and "black curly hair"
  - robocat 4y ago
    You really are not helping that cause. As a foreigner[], your point confused me anyway, and doing a Google for cultural stuff usually gets variable results. But I did laugh at many of the comments here https://www.reddit.com/r/TooAfraidToAsk/comments/ufy2k4/why_is_it_considered_offensive_for_white_people/ https://www.reddit.com/r/TooAfraidToAsk/comments/ufy2k4/why_... [] probably, New Zealand, although foreigner is relative
    - ChadNauseam 4y ago
      Haha. I've got some personal experience with that one. I used to live in a house with many other people, and one girl was rastafarian and from jamacia and had dreadlocks, and another girl in the house (who wasn't black) thought that her hairstyle was very offensive. We had to have several conflict resolution meetings about it. As silly as it seemed, I do think everyone is entitled to their own opinion and I respect the anti-dreadlocks girl for standing up for what she believed in even when most people were against her.
      - robocat 4y ago
        > thought that her hairstyle was very offensive Telling others they don’t like how others look is right near the top on the scale of offensiveness. I had a partner who had had dreads for 25 years. I’m wasn’t a huge fan of her dreads because although I like the look, hers were somewhat annoying for me (scratchy, dread babies, me getting tangled). That said, I would hope I never tell any other person how to look. Hilarious when she was working, and someone would treat her badly due to their assumptions or prejudices, only to discover to their detriment she was very senior staff! Dreadlocks are usually called dreads in NZ. My previous link mentions that some people call them locks, which seems inapproprate to me: kind of a confusing whitewashing denial of history.
barredo 4y ago
I know you're anon trolling, but the authors' names are: Chitwan Saharia, William Chan, Saurabh Saxena†, Lala Li†, Jay Whang†, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho†, David Fleet†, Mohammad Norouzi
- pid-1 4y ago
  Absolutely not related to the whole discussion, but what do "†" stands for?
  - dntrkv 4y ago
    https://en.wikipedia.org/wiki/Dagger_(mark) https://en.wikipedia.org/wiki/Dagger_(mark)
  - joshcryer 4y ago
    It's just a different asterisk to distinguish, in this case, in the paper, they are "core contributors."
- deleted 4y ago
  [deleted]
- hda2 4y ago
  Google AI researchers don't have the final say in what gets published and what doesn't. I think there was a huge controversy when people learned about it last year.
josho 4y ago
Translation: AI has the potential to transform society. When we release this model to the public it will be used in ways we haven’t anticipated. We know the model has bias and we need more time to consider releasing this to the public out of concerns that this transformative technology further perpetuate mistakes that we’ve made in our recent past.
- curiousgal 4y ago
  > it will be used in ways we haven’t anticipated Oh yeah, as a woman who grew up in a Third World country, how an AI model generates images would have deeply affected my daily struggles! /s It's kinda insulting that they think that this would be insulting. Like "Oh no I asked the model to draw a doctor and it drew a male doctor, I guess there's no point in me pursuing medical studies" ...
  - boppo1 4y ago
    I don't think the concern over offense is actually about you. There's a metagame here which is that if it could potentially offend you (third-world-originated-woman), then there's a brand-image liability for the company. I don't think they care about you, I think they care about not being hit on as "the company that algorithmically identifies black people as gorillas".
  - pxmpxm 4y ago
    Postmodernism is what postmodernism does.
    - contingencies 4y ago
      Love it. Added to https://github.com/globalcitizen/taoup https://github.com/globalcitizen/taoup
      - pxmpxm 4y ago
        Ha! However different pxmpxm on github, I'm afraid.
Ar-Curunir 4y ago
Except "reality" in this case is just their biased training set. E.g. There's more non-white doctors and nurses in the world than white ones, yet their model would likely show an image of white person when you type in "doctor".
- umeshunni 4y ago
  Alternately, there are more females nurses in the world than male nurses, and their model probably shows an image of a woman when you type in "nurse" but they consider that a problem.
  - contingencies 4y ago
    @Google Brain Toronto Team: See what you get when you generate nurses with ncurses.
  - astrange 4y ago
    Google Image Search doesn’t reflect harsh reality when you search for things; it shows you what’s on Pinterest. The same is more likely to apply here than the idea they’re trying to hide something. There’s no reason to believe their model training learns the same statistics as their input dataset even. If that’s not an explicit training goal then whatever happens happens. AI isn’t magic or more correct than people.
  - webmaven 4y ago
    > their model probably shows an image of a woman when you type in "nurse" but they consider that a problem. There is a difference between probably and invariably. Would it be so hard for the model to show male nurses at least some of the time?
JohnBooty 4y ago
Translation: we need to hand-tune this to not reflect reality Is it reflecting reality, though? Seems to me that (as with any ML stuff, right?) it's reflecting the training corpus. Futhermore, is it this thing's job to reflect reality? the world as we (Caucasian/Asian male American woke upper-middle class San Fransisco engineers) wish it to be Snarky answer: Ah, yes, let's make sure that things like "A giant cobra snake on a farm. The snake is made out of corn" reflect reality. Heartfelt answer: Yes, there is some of that wishful thinking or editorializing. I don't consider it to be erasing or denying reality. This is a tool that synthesizes unreality. I don't think that such a tool should, say, refuse to synthesize an image of a female POTUS because one hasn't existed yet. This is art, not a reporting tool... and keep in mind that art not only imitates life but also influences it.
- nomel 4y ago
  > Snarky answer: Ah, yes, let's make sure that things like "A giant cobra snake on a farm. The snake is made out of corn" reflect reality. If it didn't reflect reality, you wouldn't be impressed by the image of the snake made of corn.
  - JohnBooty 4y ago
    Pardon? The snake made of corn most certainly does not reflect reality: snakes made out of corn do not exist.
userbinator 4y ago
Indeed. As the saying goes, we are truly living in a post-truth world.
holmesworcester 4y ago
"As we wish it to be" is not totally true, because there are some places where humanity's iconographic reality (which Imagen trains on) differs significantly from actual reality. One example would be if Imagen draws a group of mostly white people when you say "draw a group of people". This doesn't reflect actual reality. Another would be if Imagen draws a group of men when you say "draw a group of doctors". In these cases where iconographic reality differs from actual reality, hand-tuning could be used to bring it closer to the real world, not just the world as we might wish it to be! I agree there's a problem here. But I'd state it more as "new technologies are being held to a vastly higher standard than existing ones." Imagine TV studios issuing a moratorium on any new shows that made being white (or rich) seem more normal than it was! The public might rightly expect studios to turn the dials away from the blatant biases of the past, but even if this would be beneficial the progressive and activist public is generations away from expecting a TV studio to not release shows until they're confirmed to be bias-free. That said, Google's decision to not publish is probably less about the inequities in AI's representation of reality and more about the AI sometimes spitting out drawings that are offensive in the US, like racist caricatures.

ceejayoz 4y ago

"Reality" as defined by the available training set isn't necessarily reality. For example, Google's image search results pre-tweaking had some interesting thoughts on what constitutes a professional hairstyle, and that searches for "men" and "women" should only return light-skinned people: https://www.theguardian.com/technology/2016/apr/08/does-google-unprofessional-hair-results-prove-algorithms-racist- https://www.theguardian.com/technology/2016/apr/08/does-goog... Does that reflect reality? No. (I suspect there are also mostly unstated but very real concerns about these being used as child pornography, revenge porn, "show my ex brutally murdered" etc. generators.)

ceeplusplus 4y ago
The reality is that hair styles on the left side of the image in the article are widely considered unprofessional in today's workplaces. That may seem egregiously wrong to you, but it is a truth of American and European society today. Should it be Google's job to rewrite reality?
- rcMgD2BwE72F 4y ago
  In any case, Google will be writing their reality. Who picked the image sample for the ML to run on, if not Google? What's the problem with writing it again, then? They know their biases and want to act on it. It's like blaming a friend for trying to phrase things nicely, and telling them to speak headlong with zero concern for others instead. Unless you believe anyone trying to do good is being hypocrite… I, for one, like civility.
- ceejayoz 4y ago
  The "unprofessional" results are almost exclusively black women; the "professional" ones are almost exclusively white or light skinned. Unless you think white women are immune to unprofessional hairstyles, and black women incapable of them, there's a race problem illustrated here even if you think the hairstyles illustrated are fairly categorized.
  - rvnx 4y ago
    If you type as a prompt "most beautiful woman in the world", you get a brown-skinned brown-haired woman with hazel eyes. What should be the right answer then ? You put a blonde, you offend the brown haired. You put blue eyes, you offend the brown eyes. etc.
    - ceejayoz 4y ago
      That's an unanswerable question. Perhaps the answer is "don't". Siri takes this approach for a wide range of queries.
      - nomel 4y ago
        How do you pick what should and shouldn't be restricted? Is there some "offense threshold"? I suspect all queries relating to religion, ethnicity, sexuality, and gender will need to be restricted, which almost certainly means you probably can't include humans at all, other than ones artificially inserted with mathematically proven random attributes. Maybe that's why none are in this demo.
        daenz 4y ago
        "Is Taiwan a country" also comes to mind.
        rvnx 4y ago
        What would a human who can freely speak without morale or being judged say on average after having ingested all the information on the internet ?
- deleted 4y ago
  [deleted]
- colinmhayes 4y ago
  Only black people have unprofessional hair and only white people have professional hair is not reality.
rvnx 4y ago
If your query was about hairstyle, why do you even look or care about the skin color ? Nowhere there is any precision for a preferred skin color in the query of th user. So it sorts and gives the most average examples based on the examples that were found on the internet. Essentially answering the query "SELECT * FROM `non-professional hairstyles` ORDER BY score DESC LIMIT 10". It's like if you search on Google "best place for wedding night". You may get 3 places out of 10 in Santorini, Greece. Yes you could have an human remove these biases because you feel that Sri Lanka is the best place for a wedding, but what if there is a consensus that Santorini is really the most appraised in the forums or websites that were crawled by Google ?
- ceejayoz 4y ago
  > The algorithm is just ranking the top "non-professional hairstyle" in the most neutral way in its database You're telling me those are all the most non-professional hairstyles available? That this is a reasonable assessment? That fairly standard, well-kept, work-appropriate curly black hair is roughly equivalent to the pink-haired, three-foot-wide hairstyle that's one of the only white people in the "unprofessional" search? Each and everyone of them is less workplace appropriate than, say, http://www.7thavenuecostumes.com/pictures/750x950/P_CC_70594.jpg http://www.7thavenuecostumes.com/pictures/750x950/P_CC_70594... ?
  - rvnx 4y ago
    I'm saying that the dataset needs to be expanded to cover the most examples possible. Work a lot on adding even more examples, in order to make the algorithms as close as possible to the "average reality". At some point we may even ultimately reach the state that the robots even collect intelligence directly in the real world, and not on the internet (even closer to reality). Censoring results sounds the best recipe for a dystopian world where only one view is right.
- jayd16 4y ago
  The results are not inherently neutral because the database is from non-neutral input. It's a simple case of sample bias.
userbinator 4y ago
unstated but very real concerns I say let people generate their own reality. The sooner the masses realise that ceci n'est pas une pipe , the less likely they are to be swayed by the growing un-reality created by companies like Google.
ChadNauseam 4y ago
You know, it wouldn't surprise me if people talking about how black curly hair shouldn't be seen as unprofessional contributed to google thinking there's an association between the concepts of "unprofessional hair" and "black curly hair"
- robocat 4y ago
  You really are not helping that cause. As a foreigner[], your point confused me anyway, and doing a Google for cultural stuff usually gets variable results. But I did laugh at many of the comments here https://www.reddit.com/r/TooAfraidToAsk/comments/ufy2k4/why_is_it_considered_offensive_for_white_people/ https://www.reddit.com/r/TooAfraidToAsk/comments/ufy2k4/why_... [] probably, New Zealand, although foreigner is relative
  - ChadNauseam 4y ago
    Haha. I've got some personal experience with that one. I used to live in a house with many other people, and one girl was rastafarian and from jamacia and had dreadlocks, and another girl in the house (who wasn't black) thought that her hairstyle was very offensive. We had to have several conflict resolution meetings about it. As silly as it seemed, I do think everyone is entitled to their own opinion and I respect the anti-dreadlocks girl for standing up for what she believed in even when most people were against her.
    - robocat 4y ago
      > thought that her hairstyle was very offensive Telling others they don’t like how others look is right near the top on the scale of offensiveness. I had a partner who had had dreads for 25 years. I’m wasn’t a huge fan of her dreads because although I like the look, hers were somewhat annoying for me (scratchy, dread babies, me getting tangled). That said, I would hope I never tell any other person how to look. Hilarious when she was working, and someone would treat her badly due to their assumptions or prejudices, only to discover to their detriment she was very senior staff! Dreadlocks are usually called dreads in NZ. My previous link mentions that some people call them locks, which seems inapproprate to me: kind of a confusing whitewashing denial of history.

barredo 4y ago

I know you're anon trolling, but the authors' names are: Chitwan Saharia, William Chan, Saurabh Saxena†, Lala Li†, Jay Whang†, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho†, David Fleet†, Mohammad Norouzi

pid-1 4y ago
Absolutely not related to the whole discussion, but what do "†" stands for?
- dntrkv 4y ago
  https://en.wikipedia.org/wiki/Dagger_(mark) https://en.wikipedia.org/wiki/Dagger_(mark)
- joshcryer 4y ago
  It's just a different asterisk to distinguish, in this case, in the paper, they are "core contributors."
deleted 4y ago
[deleted]
hda2 4y ago
Google AI researchers don't have the final say in what gets published and what doesn't. I think there was a huge controversy when people learned about it last year.

josho 4y ago

Translation: AI has the potential to transform society. When we release this model to the public it will be used in ways we haven’t anticipated. We know the model has bias and we need more time to consider releasing this to the public out of concerns that this transformative technology further perpetuate mistakes that we’ve made in our recent past.

curiousgal 4y ago
> it will be used in ways we haven’t anticipated Oh yeah, as a woman who grew up in a Third World country, how an AI model generates images would have deeply affected my daily struggles! /s It's kinda insulting that they think that this would be insulting. Like "Oh no I asked the model to draw a doctor and it drew a male doctor, I guess there's no point in me pursuing medical studies" ...
- boppo1 4y ago
  I don't think the concern over offense is actually about you. There's a metagame here which is that if it could potentially offend you (third-world-originated-woman), then there's a brand-image liability for the company. I don't think they care about you, I think they care about not being hit on as "the company that algorithmically identifies black people as gorillas".
- pxmpxm 4y ago
  Postmodernism is what postmodernism does.
  - contingencies 4y ago
    Love it. Added to https://github.com/globalcitizen/taoup https://github.com/globalcitizen/taoup
    - pxmpxm 4y ago
      Ha! However different pxmpxm on github, I'm afraid.

Ar-Curunir 4y ago

Except "reality" in this case is just their biased training set. E.g. There's more non-white doctors and nurses in the world than white ones, yet their model would likely show an image of white person when you type in "doctor".

umeshunni 4y ago
Alternately, there are more females nurses in the world than male nurses, and their model probably shows an image of a woman when you type in "nurse" but they consider that a problem.
- contingencies 4y ago
  @Google Brain Toronto Team: See what you get when you generate nurses with ncurses.
- astrange 4y ago
  Google Image Search doesn’t reflect harsh reality when you search for things; it shows you what’s on Pinterest. The same is more likely to apply here than the idea they’re trying to hide something. There’s no reason to believe their model training learns the same statistics as their input dataset even. If that’s not an explicit training goal then whatever happens happens. AI isn’t magic or more correct than people.
- webmaven 4y ago
  > their model probably shows an image of a woman when you type in "nurse" but they consider that a problem. There is a difference between probably and invariably. Would it be so hard for the model to show male nurses at least some of the time?

JohnBooty 4y ago

Translation: we need to hand-tune this to not reflect reality Is it reflecting reality, though? Seems to me that (as with any ML stuff, right?) it's reflecting the training corpus. Futhermore, is it this thing's job to reflect reality? the world as we (Caucasian/Asian male American woke upper-middle class San Fransisco engineers) wish it to be Snarky answer: Ah, yes, let's make sure that things like "A giant cobra snake on a farm. The snake is made out of corn" reflect reality. Heartfelt answer: Yes, there is some of that wishful thinking or editorializing. I don't consider it to be erasing or denying reality. This is a tool that synthesizes unreality. I don't think that such a tool should, say, refuse to synthesize an image of a female POTUS because one hasn't existed yet. This is art, not a reporting tool... and keep in mind that art not only imitates life but also influences it.

nomel 4y ago
> Snarky answer: Ah, yes, let's make sure that things like "A giant cobra snake on a farm. The snake is made out of corn" reflect reality. If it didn't reflect reality, you wouldn't be impressed by the image of the snake made of corn.
- JohnBooty 4y ago
  Pardon? The snake made of corn most certainly does not reflect reality: snakes made out of corn do not exist.

userbinator 4y ago

Indeed. As the saying goes, we are truly living in a post-truth world.

holmesworcester 4y ago

"As we wish it to be" is not totally true, because there are some places where humanity's iconographic reality (which Imagen trains on) differs significantly from actual reality. One example would be if Imagen draws a group of mostly white people when you say "draw a group of people". This doesn't reflect actual reality. Another would be if Imagen draws a group of men when you say "draw a group of doctors". In these cases where iconographic reality differs from actual reality, hand-tuning could be used to bring it closer to the real world, not just the world as we might wish it to be! I agree there's a problem here. But I'd state it more as "new technologies are being held to a vastly higher standard than existing ones." Imagine TV studios issuing a moratorium on any new shows that made being white (or rich) seem more normal than it was! The public might rightly expect studios to turn the dials away from the blatant biases of the past, but even if this would be beneficial the progressive and activist public is generations away from expecting a TV studio to not release shows until they're confirmed to be bias-free. That said, Google's decision to not publish is probably less about the inequities in AI's representation of reality and more about the AI sometimes spitting out drawings that are offensive in the US, like racist caricatures.