7 ms·

Google Scrambles After Software IDs Photo of Two Black People as “Gorillas”

21 points by lingben 11y ago

Lawtonfogle 11y ago
So assuming this was a mistaken classification that happened naturally and not the result of someone purposefully training the system to do this... does this mean we need to teach these systems to be politically correct? Confusing a dog and a goat doesn't carry the same potential to cause extreme negative publicity that an error like this does, especially when the system works well enough that most other classification attempts are correct.
- SmokyBourbon 11y ago
  $ npm install socialjustice@0.2.1 npm http GET https://reddit.com/censorship https://reddit.com/censorship npm http GET https://anitasarkeesian.com/triggers https://anitasarkeesian.com/triggers npm http GET https://huffingtonpost.com/secretmailinglist https://huffingtonpost.com/secretmailinglist Checking privilege.... npm !ERR white males found npm !ERR not ok code 0 $
- fwn 11y ago
  We are affected by this mistake because we are socialised to be offended and outraged by labeling we perceive as offensive. We do this to substitute an impossible external enforcer (of common norms) with an internal, distributed one. It is tough, emotional work to understand that there is no need to punish an algorithm. I guess it takes time and additional norms (e.g. devs saying they are sorry, etc..) to adapt to this new existence of semiautomatic affronts.
- minikites 11y ago
  I think a possible root to the problem is the general lack of diversity in tech. A bug that confuses white people for goats will be caught sooner than a bug that mistakes black people for gorillas because there are more white people around to notice that problem. It wouldn't surprise me if the initial test data included few to no non-white people.
johansch 11y ago
"Like I understand HOW this happens; the problem is moreso on the WHY" Eh, what?
- hoverbear 11y ago
  They apparently are clever enough to understand the identification algorithm, but not the social implications.
PaulHoule 11y ago
If I called somebody a gorilla out of anger, I might end up apologizing to the gorilla. Gorillas are much better behaved than primates in general so I wouldn't be insulted if somebody called me a gorilla.
- mikeyouse 11y ago
  > Gorillas are much better behaved than primates in general so I wouldn't be insulted if somebody called me a gorilla. Then again, you likely don't belong to a race that was subjugated for centuries by people comparing them to monkeys or gorillas to intentionally dehumanize them.. Black athletes in foreign countries still get bananas thrown at them on the field. I can assure you, this isn't meant as a compliment. Eberhardt described an experiment based on the Rodney King case. Participants viewed a video of police officers beating a Black suspect. Participants who saw words associated with apes before watching the video believed the officers’ actions were more justified, compared to participants who did not see the animal imagery. In a separate study, Eberhardt found that people are more likely to consider animal language (such as “barbaric,” “animalistic,” etc.) appropriate and necessary in court cases of Black defendants compared to White defendants. News articles describing Black defendants are more likely to use animal language than articles about White defendants. In addition, news articles of Blacks who received a death sentence contain more animal language than the articles of those with life sentences. “So not only are Blacks associated with apes, but this association is linked to justifications of violence and death,” Eberhardt concluded. “It’s almost as though the rules for what moral treatment is get shifted for Black suspects and defendants.” http://gender.stanford.edu/news/2011/continued-dehumanization-blacks http://gender.stanford.edu/news/2011/continued-dehumanizatio...
  - PaulHoule 11y ago
    I'd say say that homo sap is on the list of worse behaved apes than Gorillas. Chimpanzees and Gibbons will tear off your face, but it is a big mistake to suppose animals are in a lower ethical state than humans, as animals don't * have 20 cops beat up one suspect * deal crack * attack people with assault rifles out of hate * invent poison gas * find demeaning labels for one another, etc. Look at animal behavior and it seems that animals are basically rational in that they do things in the social space that raise their utility function. This is not so clear in the case of humans (an herbivore will always express satisfaction if you feed it, a human might not.)
civilian 11y ago
I'm curious what white people are mistaken for when you have close ups of their faces. Potatoes?
l-jenkins 11y ago
This is the problem with automatic image processing and categorization. It's good, but mistakes like this can happen and embarrass an entire company.
- DevPad 11y ago
  Yeah, shit happens sometimes. Especially for AI algos.
DanBC 11y ago
> Ultimately, Google applied a secondary solution, reworking the system so that it wouldn’t tie photos to the “Gorilla” tag at all. Zunger writes that it is also working to develop a number of “longer-term fixes,” including identifying “words to be careful about in photos of people” and “better recognition of dark skinned faces.” I hope photos of gorillas are not now marked as "black people" - since they've blocked the gorilla tag being used.
- DevPad 11y ago
  > Ultimately, Google applied a secondary solution, reworking the system so that it wouldn’t tie photos to the “Gorilla” tag at all Looks like dirty fix.
pvaldes 11y ago
Great algorithm accuracy: 98,25% of DNA shared between gorillas and humans. I could forgiven the error easily is someone tagged me as gorilla. Modify tags with positive adjectives that work both ways like "Strong as a gorilla" or "Happy as a chimp" and the 90% of the current problems will dissapear.

21 points by lingben 11y ago

So assuming this was a mistaken classification that happened naturally and not the result of someone purposefully training the system to do this... does this mean we need to teach these systems to be politically correct? Confusing a dog and a goat doesn't carry the same potential to cause extreme negative publicity that an error like this does, especially when the system works well enough that most other classification attempts are correct.

SmokyBourbon 11y ago
$ npm install socialjustice@0.2.1 npm http GET https://reddit.com/censorship https://reddit.com/censorship npm http GET https://anitasarkeesian.com/triggers https://anitasarkeesian.com/triggers npm http GET https://huffingtonpost.com/secretmailinglist https://huffingtonpost.com/secretmailinglist Checking privilege.... npm !ERR white males found npm !ERR not ok code 0 $
fwn 11y ago
We are affected by this mistake because we are socialised to be offended and outraged by labeling we perceive as offensive. We do this to substitute an impossible external enforcer (of common norms) with an internal, distributed one. It is tough, emotional work to understand that there is no need to punish an algorithm. I guess it takes time and additional norms (e.g. devs saying they are sorry, etc..) to adapt to this new existence of semiautomatic affronts.
minikites 11y ago
I think a possible root to the problem is the general lack of diversity in tech. A bug that confuses white people for goats will be caught sooner than a bug that mistakes black people for gorillas because there are more white people around to notice that problem. It wouldn't surprise me if the initial test data included few to no non-white people.

johansch 11y ago

"Like I understand HOW this happens; the problem is moreso on the WHY" Eh, what?

hoverbear 11y ago
They apparently are clever enough to understand the identification algorithm, but not the social implications.

PaulHoule 11y ago

If I called somebody a gorilla out of anger, I might end up apologizing to the gorilla. Gorillas are much better behaved than primates in general so I wouldn't be insulted if somebody called me a gorilla.

mikeyouse 11y ago
> Gorillas are much better behaved than primates in general so I wouldn't be insulted if somebody called me a gorilla. Then again, you likely don't belong to a race that was subjugated for centuries by people comparing them to monkeys or gorillas to intentionally dehumanize them.. Black athletes in foreign countries still get bananas thrown at them on the field. I can assure you, this isn't meant as a compliment. Eberhardt described an experiment based on the Rodney King case. Participants viewed a video of police officers beating a Black suspect. Participants who saw words associated with apes before watching the video believed the officers’ actions were more justified, compared to participants who did not see the animal imagery. In a separate study, Eberhardt found that people are more likely to consider animal language (such as “barbaric,” “animalistic,” etc.) appropriate and necessary in court cases of Black defendants compared to White defendants. News articles describing Black defendants are more likely to use animal language than articles about White defendants. In addition, news articles of Blacks who received a death sentence contain more animal language than the articles of those with life sentences. “So not only are Blacks associated with apes, but this association is linked to justifications of violence and death,” Eberhardt concluded. “It’s almost as though the rules for what moral treatment is get shifted for Black suspects and defendants.” http://gender.stanford.edu/news/2011/continued-dehumanization-blacks http://gender.stanford.edu/news/2011/continued-dehumanizatio...
- PaulHoule 11y ago
  I'd say say that homo sap is on the list of worse behaved apes than Gorillas. Chimpanzees and Gibbons will tear off your face, but it is a big mistake to suppose animals are in a lower ethical state than humans, as animals don't * have 20 cops beat up one suspect * deal crack * attack people with assault rifles out of hate * invent poison gas * find demeaning labels for one another, etc. Look at animal behavior and it seems that animals are basically rational in that they do things in the social space that raise their utility function. This is not so clear in the case of humans (an herbivore will always express satisfaction if you feed it, a human might not.)

civilian 11y ago

I'm curious what white people are mistaken for when you have close ups of their faces. Potatoes?

l-jenkins 11y ago

This is the problem with automatic image processing and categorization. It's good, but mistakes like this can happen and embarrass an entire company.

DevPad 11y ago
Yeah, shit happens sometimes. Especially for AI algos.

DanBC 11y ago

> Ultimately, Google applied a secondary solution, reworking the system so that it wouldn’t tie photos to the “Gorilla” tag at all. Zunger writes that it is also working to develop a number of “longer-term fixes,” including identifying “words to be careful about in photos of people” and “better recognition of dark skinned faces.” I hope photos of gorillas are not now marked as "black people" - since they've blocked the gorilla tag being used.

DevPad 11y ago
> Ultimately, Google applied a secondary solution, reworking the system so that it wouldn’t tie photos to the “Gorilla” tag at all Looks like dirty fix.

pvaldes 11y ago

Great algorithm accuracy: 98,25% of DNA shared between gorillas and humans. I could forgiven the error easily is someone tagged me as gorilla. Modify tags with positive adjectives that work both ways like "Strong as a gorilla" or "Happy as a chimp" and the 90% of the current problems will dissapear.