4 ms·

What in the world is going on with these convoluted framings? Just call it racism - not a 'shortcut' or an 'overcorrection'. Why not just clearly, simply, and p

What in the world is going on with these convoluted framings? Just call it racism - not a 'shortcut' or an 'overcorrection'. Why not just clearly, simply, and precisely describe what is going on? This is anti-white racism. Google Gemini was erasing white people from history and also the present, and when it wasn't simply erasing them, it was diluting them - going partway towards erasure. What I've said is bare, basic truth, and not ideological. If it offends or provokes factual or semantic quibbling, then such reaction indicates pretzel thinking about moral good and 'racism' in the minds of people like Smith.

ForHackernews 3y ago
Is it 'racism' when facial recognition algorithms perform more poorly on dark skin? It seems like a technical glitch to me. Similarly here, they want the models to represent ethnically diverse individuals, but the model doesn't understand historical context well. It's a technical shortcoming.
- anonporridge 3y ago
  > Is it 'racism' when facial recognition algorithms perform more poorly on dark skin? No, because that occurred due to an oversight of not including a representative set of training data. At worst it's implicit racism due to oversight and ignorance. What Google seemed to do here was more explicit racism, by setting configuration in their system that directly and intentionally discriminates based on race. It's fundamentally worse.
  - ForHackernews 3y ago
    They both seem like goofs to me. I don't see much qualitative distinction. In both cases, engineers have produced racially dubious results by getting sloppy or lazy with their AI models. Personally, I wouldn't call that "racism" but I see how one could make that argument.
    - anonporridge 3y ago
      Is it not obvious that the distinction is intent to discriminate based on race? Google, in a self imposed noble desire to rectify poor representation of training data, intentionally injected race discrimination in their system. This intent is what makes it fundamentally worse than unintentionally having non diverse training data.
    - adamsb6 3y ago
      They created a mirror. They didn’t like that the mirror showed reality, so they made it into a one way race-changing funhouse mirror.
    - baobabKoodaa 3y ago
      If you read the OP you will find a link to Google statement where Google officially admits they did this intentionally. They tried to shoot an arrow, and they "missed the mark", oops. They're not sorry for shooting the arrow, they are sorry for missing the mark. Nobody who worked on facial recognition that failed to recognize black people, nobody in that team was intentionally trying to fail on black people. They didn't try to shoot an arrow, although they accidentally shot one in their own leg. See how these things are different?
- xepriot 3y ago
  No, when unintentioned outcomes occur in the world, this is not racism. The sun is not racist for burning white skin more severely than dark skin, and when AI are trained on disproportionately white data, their outputs are not racism. When racist employees of Google intentionally racially cleanse their products, this is racism. Let me know what is confusing about the concepts of agency and intention., and I will try to clarify
  - ForHackernews 3y ago
    "I want a model that recognizes faces, but I didn't train it on enough black faces" => not-racism, according to you. "I want a model that produces racially diverse images of people, but I didn't test it in racially homogenous historical contexts" => yes-racism, according to you.
    - xepriot 3y ago
      Did the developers intend their product to manipulate race in its outputs or not? It's a simpler question than these contorted reformulations you present. But probably not simple enough to pierce ideological bias. I'll ask instead: you really believe that it wasn't tested in historical and other contexts where white people were erased?
      - ForHackernews 3y ago
        > I'll ask instead: you really believe that it wasn't tested in historical and other contexts where white people were erased? Occam's Razor: Of course it wasn't or they wouldn't have released such a ridiculous embarrassing product! C'mon, I think you're the one wearing blinders here.
        baobabKoodaa 3y ago
        > Of course it wasn't or they wouldn't have released such a ridiculous embarrassing product! Then please do explain why their first reaction was to double down on "diversity" in historical contexts, before eventually retreating?
- choeger 3y ago
  It could be a consequence of racism. Racism lead to white people dominating the United States, which lead to images of white people dominating the internet, which lead to images of white people dominating the training data for facial recognition algorithms. But it could also just reflect the composition of the population of the countries that dominate the internet.
  - spidersenses 3y ago
    [flagged]
    - polotics 3y ago
      You're looking at adverts trying to get an edge of memorability by triggering cognitive dissonance the way they know will on the German general population...
      - spidersenses 3y ago
        That would be a very positive interpretation of the intention behind the strategy. Who would really buy a product where the manufacturer tries to provoke them? For a historical example have a look at Gillette's failed marketing campaign from a few years ago: https://time.com/5503156/gillette-razors-toxic-masculinity/ https://time.com/5503156/gillette-razors-toxic-masculinity/
    - tptacek 3y ago
      I guess it depends on how much they're paying the mixed-race babies.
- deleted 3y ago
  [deleted]
- archsurface 3y ago
  No, this is not limited to gemini.
- lolinder 3y ago
  Neither case is a purely technical shortcoming—in both the case of facial recognition algorithms performing poorly on dark skin and the case of Gemini refusing to produce images of white people, the technology works that way because of the priorities of the people developing the software, and we can derive the developers' priorities from the software's limitations. A piece of facial recognition software that doesn't do well with dark skin shows that the team that built it either didn't think to test it on dark skin or didn't feel it was important to get dark skin working before release. An image generation model that both doesn't produce images of white people spontaneously and will actively refuse to do so when asked shows that the team that built it either failed to ever ask it to produce images of white people or believed that it was a good thing for their model to refuse to produce such images. In neither case is the problem purely technical.
anonporridge 3y ago
Because there's a newspeak movement to try to dilute the definition of 'racism' as something that can't be perpetrated against relatively powerful racial groups.
deleted 3y ago
[deleted]
mrkramer 3y ago
Google overengineers itself over and over again, liberal managers are dangerous. Make generative AI that is grounded in historical facts and let people play with it. For example if all popes were white, generate white pope image but let me also generate black pope because why not, if technology can do it.
at_a_remove 3y ago
Clearly, you've never spent any time on Metafilter. There's been a long push to reformulate the concept of racism such that white people cannot be a victim of it. Even in an environment with a reversal of the usual population statistics. Similarly, cannot be sexist against men, and so on. Right now, the DEI exists in these models at a level where context is not considered. Not for time or locale, nor for, say, the correlation between sex and different populations present in a given profession. It's just an unconsidered DEI goo sprayed over the outputs.
lolinder 3y ago
The convoluted framings come from people who want to actually persuade those who don't already agree with them. If you want someone to listen to you you have to first meet them where they are, not because you're worried about offense but because you want them to change their mind more than you want to state "bare, basic truth". The people behind Gemini's "diversity" sincerely believe they're doing the right thing. Calling them racist doesn't help change their mind, it just gets you filed under "reactionary" and ignored. This piece takes a more effective approach by saying "I'm with you, I think you want to do the right thing, but here's why what you're doing is actually counterproductive". It's an effective rhetorical strategy for the target audience, you're just not that audience because you already see the problem.
- xepriot 3y ago
  This is certainly charitable to Noah Smith, but I think it's more likely he earnestly has the overcomplicated understanding of racism he appears to demonstrate in his article. And it does have the same effect as if he did: he advances the confused understanding of racism as something that only happens to non-whites. Of course everyone has good intentions and likes to be flattered. It's possible to flatter with tone, and without dissembling.
  - lolinder 3y ago
    Yes, in order to convincingly say "I'm with you, I think you want to do the right thing" you do have to actually believe that America has a race problem that needs to be addressed and you have to believe that the people taking shortcuts sincerely believe they're doing the right thing. So yeah, Noah Smith does presumably hold that understanding of racism. I wasn't trying to say that he was lying for the sake of the article, I was trying to say that he's framing it in a way that is intended to be persuasive to a particular group of people that you're not a part of. If you pressed him on it in a private conversation I wouldn't be surprised if he acknowledged that the behaviors involved here are racist (by some definition) even though the people are trying to be the opposite. But he knows better than to level that word at his audience.
    - 3y ago

ForHackernews 3y ago

Is it 'racism' when facial recognition algorithms perform more poorly on dark skin? It seems like a technical glitch to me. Similarly here, they want the models to represent ethnically diverse individuals, but the model doesn't understand historical context well. It's a technical shortcoming.

anonporridge 3y ago
> Is it 'racism' when facial recognition algorithms perform more poorly on dark skin? No, because that occurred due to an oversight of not including a representative set of training data. At worst it's implicit racism due to oversight and ignorance. What Google seemed to do here was more explicit racism, by setting configuration in their system that directly and intentionally discriminates based on race. It's fundamentally worse.
- ForHackernews 3y ago
  They both seem like goofs to me. I don't see much qualitative distinction. In both cases, engineers have produced racially dubious results by getting sloppy or lazy with their AI models. Personally, I wouldn't call that "racism" but I see how one could make that argument.
  - anonporridge 3y ago
    Is it not obvious that the distinction is intent to discriminate based on race? Google, in a self imposed noble desire to rectify poor representation of training data, intentionally injected race discrimination in their system. This intent is what makes it fundamentally worse than unintentionally having non diverse training data.
  - adamsb6 3y ago
    They created a mirror. They didn’t like that the mirror showed reality, so they made it into a one way race-changing funhouse mirror.
  - baobabKoodaa 3y ago
    If you read the OP you will find a link to Google statement where Google officially admits they did this intentionally. They tried to shoot an arrow, and they "missed the mark", oops. They're not sorry for shooting the arrow, they are sorry for missing the mark. Nobody who worked on facial recognition that failed to recognize black people, nobody in that team was intentionally trying to fail on black people. They didn't try to shoot an arrow, although they accidentally shot one in their own leg. See how these things are different?
xepriot 3y ago
No, when unintentioned outcomes occur in the world, this is not racism. The sun is not racist for burning white skin more severely than dark skin, and when AI are trained on disproportionately white data, their outputs are not racism. When racist employees of Google intentionally racially cleanse their products, this is racism. Let me know what is confusing about the concepts of agency and intention., and I will try to clarify
- ForHackernews 3y ago
  "I want a model that recognizes faces, but I didn't train it on enough black faces" => not-racism, according to you. "I want a model that produces racially diverse images of people, but I didn't test it in racially homogenous historical contexts" => yes-racism, according to you.
  - xepriot 3y ago
    Did the developers intend their product to manipulate race in its outputs or not? It's a simpler question than these contorted reformulations you present. But probably not simple enough to pierce ideological bias. I'll ask instead: you really believe that it wasn't tested in historical and other contexts where white people were erased?
    - ForHackernews 3y ago
      > I'll ask instead: you really believe that it wasn't tested in historical and other contexts where white people were erased? Occam's Razor: Of course it wasn't or they wouldn't have released such a ridiculous embarrassing product! C'mon, I think you're the one wearing blinders here.
      - baobabKoodaa 3y ago
        > Of course it wasn't or they wouldn't have released such a ridiculous embarrassing product! Then please do explain why their first reaction was to double down on "diversity" in historical contexts, before eventually retreating?
choeger 3y ago
It could be a consequence of racism. Racism lead to white people dominating the United States, which lead to images of white people dominating the internet, which lead to images of white people dominating the training data for facial recognition algorithms. But it could also just reflect the composition of the population of the countries that dominate the internet.
- spidersenses 3y ago
  [flagged]
  - polotics 3y ago
    You're looking at adverts trying to get an edge of memorability by triggering cognitive dissonance the way they know will on the German general population...
    - spidersenses 3y ago
      That would be a very positive interpretation of the intention behind the strategy. Who would really buy a product where the manufacturer tries to provoke them? For a historical example have a look at Gillette's failed marketing campaign from a few years ago: https://time.com/5503156/gillette-razors-toxic-masculinity/ https://time.com/5503156/gillette-razors-toxic-masculinity/
  - tptacek 3y ago
    I guess it depends on how much they're paying the mixed-race babies.
deleted 3y ago
[deleted]
archsurface 3y ago
No, this is not limited to gemini.
lolinder 3y ago
Neither case is a purely technical shortcoming—in both the case of facial recognition algorithms performing poorly on dark skin and the case of Gemini refusing to produce images of white people, the technology works that way because of the priorities of the people developing the software, and we can derive the developers' priorities from the software's limitations. A piece of facial recognition software that doesn't do well with dark skin shows that the team that built it either didn't think to test it on dark skin or didn't feel it was important to get dark skin working before release. An image generation model that both doesn't produce images of white people spontaneously and will actively refuse to do so when asked shows that the team that built it either failed to ever ask it to produce images of white people or believed that it was a good thing for their model to refuse to produce such images. In neither case is the problem purely technical.

anonporridge 3y ago

Because there's a newspeak movement to try to dilute the definition of 'racism' as something that can't be perpetrated against relatively powerful racial groups.

deleted 3y ago

[deleted]

mrkramer 3y ago

Google overengineers itself over and over again, liberal managers are dangerous. Make generative AI that is grounded in historical facts and let people play with it. For example if all popes were white, generate white pope image but let me also generate black pope because why not, if technology can do it.

at_a_remove 3y ago

Clearly, you've never spent any time on Metafilter. There's been a long push to reformulate the concept of racism such that white people cannot be a victim of it. Even in an environment with a reversal of the usual population statistics. Similarly, cannot be sexist against men, and so on. Right now, the DEI exists in these models at a level where context is not considered. Not for time or locale, nor for, say, the correlation between sex and different populations present in a given profession. It's just an unconsidered DEI goo sprayed over the outputs.

lolinder 3y ago

The convoluted framings come from people who want to actually persuade those who don't already agree with them. If you want someone to listen to you you have to first meet them where they are, not because you're worried about offense but because you want them to change their mind more than you want to state "bare, basic truth". The people behind Gemini's "diversity" sincerely believe they're doing the right thing. Calling them racist doesn't help change their mind, it just gets you filed under "reactionary" and ignored. This piece takes a more effective approach by saying "I'm with you, I think you want to do the right thing, but here's why what you're doing is actually counterproductive". It's an effective rhetorical strategy for the target audience, you're just not that audience because you already see the problem.

xepriot 3y ago
This is certainly charitable to Noah Smith, but I think it's more likely he earnestly has the overcomplicated understanding of racism he appears to demonstrate in his article. And it does have the same effect as if he did: he advances the confused understanding of racism as something that only happens to non-whites. Of course everyone has good intentions and likes to be flattered. It's possible to flatter with tone, and without dissembling.
- lolinder 3y ago
  Yes, in order to convincingly say "I'm with you, I think you want to do the right thing" you do have to actually believe that America has a race problem that needs to be addressed and you have to believe that the people taking shortcuts sincerely believe they're doing the right thing. So yeah, Noah Smith does presumably hold that understanding of racism. I wasn't trying to say that he was lying for the sake of the article, I was trying to say that he's framing it in a way that is intended to be persuasive to a particular group of people that you're not a part of. If you pressed him on it in a private conversation I wouldn't be surprised if he acknowledged that the behaviors involved here are racist (by some definition) even though the people are trying to be the opposite. But he knows better than to level that word at his audience.
  - 3y ago