4 ms·

How is that racist? Because you’re projecting a racist comment onto it, a classifier? That does not make sense to me

deleted 3y ago
[deleted]
giraffe_lady 3y ago
Racism isn't an objective order existing in the universe separate from us. It's part of human experience and exists where humans experience it. Given the recent history of equating black people with non-human primates, and using that to deny them rights & full participation in society, making this error is going to be experienced as racist. It's not a matter of individual malice or taxonomic classification, but of history and social relations.
- atonse 3y ago
  I think we can all agree that the classifier is horribly broken. But it seems like if nobody is working on this, how will we ever fix this gaping hole in image classifiers? And don't we want to fix it? And to fix it, research will continue to get it wrong until they get it less wrong and more right, but can only iterate without a massive backlash. It seems like being stuck between a rock and a hard place. I am rhetorically asking, wouldn't we have to allow researchers to iterate on this problem to fix it? That simply won't happen until we are able to allow them leeway understanding that this is an incrementally improving model. Otherwise what we have is just a sledgehammer solution (just banning all primate classifications) which actually never addressed the problem, that these models do have a race-based bias (probably in their input datasets.)
  - giraffe_lady 3y ago
    I'm simply answering the question of how it is racist, not currently trying to tackle the appropriateness of fixing the racism or the technical hurdles involved in that. It's outside my expertise and not particularly relevant to the comment I was responding to.
- 1123581321 3y ago
  I suppose this could be an example of Popper’s third world.
rjh29 3y ago
The racism comes from the fact non-white people were not properly considered when the model was developed and trained. This comes up time and time again in AI, ranging from face ID that only works on white people, to porn classifiers that associate black people with NSFW images.
- creato 3y ago
  No matter how good or well trained on good data with good representation of all skin colors a classifier is, it’s going to misclassify people and things periodically, and it’s definitely going to misclassify black people as gorillas more often than other races.
- changoplatanero 3y ago
  But, white people get misclassified as animals by the classifier too. Typically white people aren't misclassified as gorillas but as other animals. So i don't think the cause is as simple as non-white people not being considered during training.
  - rjh29 3y ago
    It classified 80 photos of the same black person as 'gorilla', I have not heard of that happening with white people.
    - changoplatanero 3y ago
      I saw lots of examples of white children being classified as seals
      - bewaretheirs 3y ago
        Have a link to an example or two? I can't find any after a few minutes of searching.
        changoplatanero 3y ago
lisasays 3y ago
If a system consistently misclassifies persons black persons far more than white persons -- and does so in a way that's obviously provocative and offensive -- then by definition it's racist in its effect (regardless of intent). The fact that the smartest company in the world cannot seem to get a handle on this problem after 8 years is also not unreasonable grounds to suspect that something's up. Like that they don't appreciate the gravity of the problem, for example.
- xeromal 3y ago
  These are dangerous grounds to discuss, but I don't think it's racist (colloquially) at all. If gorillas were like yetis and covered in white fur and it started labeling anglos as gorillas, it's not racist either. Racism (colloquially) comes from bad people's intentions. Who would've thought that a creature that is very similar to us humans and has a color that matches some humans would accidentally classify something poorly. What would be racist from this outcome is if it kept doing this and no one did anything. Clearly it hurts people's feelings and that is a very valid issue. Googles option to just nuke it is a great start until they can hammer out the kinks.
  - lisasays 3y ago
    Racism (colloquially) comes from bad people's intentions. Racism can also be measured by its effect, regardless of intent. What would be racist from this outcome is if it kept doing this and no one did anything. After 8 years, that's seems to be precisely what's happening.
    - xeromal 3y ago
      Isn't the point of the article that it just refuses to recognize gorillas outright? That prevents exactly what you're talking about. And I made that point in my post. It is hurtful so Google prevent google photos from classifying anything as a Gorilla is a good bandaid. Some things are just too risky to solve for little gain.
Spivak 3y ago
Because racism is about harm, not an estimation of a thing's motivations and prejudices -- which it's why it's still racist even if you didn't mean it or didn't know. It doesn't actually require a mind at all. Anything that confers, amplifies, or perpetuates harmful stereotypes or negative associations with people of a specific race is racist. The thing you're calling racism is actually hate speech as it's typically defined in law.

deleted 3y ago

[deleted]

Racism isn't an objective order existing in the universe separate from us. It's part of human experience and exists where humans experience it. Given the recent history of equating black people with non-human primates, and using that to deny them rights & full participation in society, making this error is going to be experienced as racist. It's not a matter of individual malice or taxonomic classification, but of history and social relations.

atonse 3y ago
I think we can all agree that the classifier is horribly broken. But it seems like if nobody is working on this, how will we ever fix this gaping hole in image classifiers? And don't we want to fix it? And to fix it, research will continue to get it wrong until they get it less wrong and more right, but can only iterate without a massive backlash. It seems like being stuck between a rock and a hard place. I am rhetorically asking, wouldn't we have to allow researchers to iterate on this problem to fix it? That simply won't happen until we are able to allow them leeway understanding that this is an incrementally improving model. Otherwise what we have is just a sledgehammer solution (just banning all primate classifications) which actually never addressed the problem, that these models do have a race-based bias (probably in their input datasets.)
- giraffe_lady 3y ago
  I'm simply answering the question of how it is racist, not currently trying to tackle the appropriateness of fixing the racism or the technical hurdles involved in that. It's outside my expertise and not particularly relevant to the comment I was responding to.
1123581321 3y ago
I suppose this could be an example of Popper’s third world.

rjh29 3y ago

The racism comes from the fact non-white people were not properly considered when the model was developed and trained. This comes up time and time again in AI, ranging from face ID that only works on white people, to porn classifiers that associate black people with NSFW images.

creato 3y ago
No matter how good or well trained on good data with good representation of all skin colors a classifier is, it’s going to misclassify people and things periodically, and it’s definitely going to misclassify black people as gorillas more often than other races.
changoplatanero 3y ago
But, white people get misclassified as animals by the classifier too. Typically white people aren't misclassified as gorillas but as other animals. So i don't think the cause is as simple as non-white people not being considered during training.
- rjh29 3y ago
  It classified 80 photos of the same black person as 'gorilla', I have not heard of that happening with white people.
  - changoplatanero 3y ago
    I saw lots of examples of white children being classified as seals
    - bewaretheirs 3y ago
      Have a link to an example or two? I can't find any after a few minutes of searching.
      - changoplatanero 3y ago

lisasays 3y ago

If a system consistently misclassifies persons black persons far more than white persons -- and does so in a way that's obviously provocative and offensive -- then by definition it's racist in its effect (regardless of intent). The fact that the smartest company in the world cannot seem to get a handle on this problem after 8 years is also not unreasonable grounds to suspect that something's up. Like that they don't appreciate the gravity of the problem, for example.

xeromal 3y ago
These are dangerous grounds to discuss, but I don't think it's racist (colloquially) at all. If gorillas were like yetis and covered in white fur and it started labeling anglos as gorillas, it's not racist either. Racism (colloquially) comes from bad people's intentions. Who would've thought that a creature that is very similar to us humans and has a color that matches some humans would accidentally classify something poorly. What would be racist from this outcome is if it kept doing this and no one did anything. Clearly it hurts people's feelings and that is a very valid issue. Googles option to just nuke it is a great start until they can hammer out the kinks.
- lisasays 3y ago
  Racism (colloquially) comes from bad people's intentions. Racism can also be measured by its effect, regardless of intent. What would be racist from this outcome is if it kept doing this and no one did anything. After 8 years, that's seems to be precisely what's happening.
  - xeromal 3y ago
    Isn't the point of the article that it just refuses to recognize gorillas outright? That prevents exactly what you're talking about. And I made that point in my post. It is hurtful so Google prevent google photos from classifying anything as a Gorilla is a good bandaid. Some things are just too risky to solve for little gain.

Spivak 3y ago

Because racism is about harm, not an estimation of a thing's motivations and prejudices -- which it's why it's still racist even if you didn't mean it or didn't know. It doesn't actually require a mind at all. Anything that confers, amplifies, or perpetuates harmful stereotypes or negative associations with people of a specific race is racist. The thing you're calling racism is actually hate speech as it's typically defined in law.