7 ms·

> Database contains photos of half of US adults without consent, and algorithm is wrong nearly 15% of time and is more likely to misidentify black people I thi

by Eyas 10y ago

> Database contains photos of half of US adults without consent, and algorithm is wrong nearly 15% of time and is more likely to misidentify black people I think technology is often a very effective way to describe what "systemic racism" is. Racism propagates easily, and race-blind / race-agnostic / race-indifferent behavior does not combat it or counteract it. Imagine a majority-white tech team, largely color-blind, yet still (by virtue of economic and geographic segregation) in a mostly white-centric bubble. The training set might be largely white. If some preprocessing or feature-detection is taking place on the raw images, it might be tracking features that could have higher variability among white people while overlooking other features. If you don't know to go out of your way to check for, include tests, and give thought to mitigating imbalances caused by race (because you "see no race") then you allow this problem to propagate. In the case of computer vision, you will inherit the "racism" of every library you import from. This reminds me of gender biased found in Word2vec[1]. It is hard to accuse the dataset of being sexist, especially when viewing it as a reflection of language in news article. Yet, people using Word2vec use it as a representative of language, not a representative of language in news, with all the baggage in today's (and past) culture. As a result, sexism can propagate, autocomplete and query suggestions might be biased. Then spending habits might be biased, etc. [1]: https://www.technologyreview.com/s/602025/how-vector-space-mathematics-reveals-the-hidden-sexism-in-language/

abandonliberty 10y ago
That is a very interesting point regarding Word2vec, but it's inappropriate to call this example racism based on the evidence provided. Only 16% of the US population is Black, while 70% is white. If the error rates between Caucasians and Blacks were the same that would, in fact, be racist - or at the very least undemocratic. It's like asking that everything in the world be symmetric when only 10% of the world is left handed.
- Eyas 10y ago
  I don't see why being in a minority means accepting a higher chance of being confused by a criminal. This is how cycles of oppression propagate.
  - eximius 10y ago
    Those aren't equivalent unless humans are out of the loop entirely, which they shouldn't be in any area of law.
    - Eyas 10y ago
      But then the humans can't be "color blind". They must explicitly go out of their way to undo the effects of the rest of the system by, E.g. Displaying extra care when dealing with a black suspect.
      - eximius 10y ago
        So, the usual, then?
  - abandonliberty 10y ago
    Your chance of getting confused BY a criminal is probably more connected to your ability to comprehend and communicate. Perhaps you not understand the metaphor. Developing takes time and money to produce accuracy. You're advocating greater per-capita investment for an ethnic group. That's racial discrimination. You are advocating racism. It should come as no surprise that there's a difference. It is therefore suspect that they avoid mentioning the magnitude of the difference; it must not support their case otherwise they would highlight it. Critical reading, logic, and reason will help protect you from this type of manipulation and inadvertent racism.
- anigbrowl 10y ago
  Your conclusion doesn't follow from your premise, which leads me to wonder whether you understand your own argument.
rhcom2 10y ago
Reminds me of the soap dispenser that doesn't detect black skin. [1] 1: https://mic.com/articles/124899/the-reason-this-racist-soap-dispenser-doesn-t-work-on-black-skin#.yNg8U7lh4 https://mic.com/articles/124899/the-reason-this-racist-soap-...
- chillacy 10y ago
  Or cars and airbags being tested on male dummies for the longest time, resulting in women (who are shorter and weigh less) getting seriously injured/killed at higher rates in accidents: https://www.bloomberg.com/view/articles/2012-08-22/why-carmakers-always-insisted-on-male-crash-test-dummies https://www.bloomberg.com/view/articles/2012-08-22/why-carma... > In some cases, according to tests with female mannequins, small women were almost three times as likely as their average male counterparts to be seriously injured or killed. A study of actual crashes by the University of Virginia’s Center for Applied Biomechanics found that women wearing seatbelts were 47 percent more likely to be seriously injured than males in similar accidents.
- bitwize 10y ago
  There was a computer system featured once on Beyond 2000 that could recognize and mimic your facial expression. But it was developed by Japanese and tested only on Japanese people. The host had to wear a black wig to make it work.
- gleenn 10y ago
  The Xbox camera also had trouble tracking African Americans.
jplasmeier 10y ago
Facial unlocking programs have this problem too. Just search for "facial unlock racism" and there's pages of results from various sources and years. Software is subject to the same effects institutions of racism, sexism, etc. and is a good example of just how pervasive these institutions are.
sundvor 10y ago
In my naive mind, I can't help but think it might simply be a case of the contrast ratio being greater in lighter skinned people - therefore making the hit ratios higher. Greater contrast ratios will make facial features stand out more.
- handedness 10y ago
  Exactly my thinking. I worked with a company that developed on a medical device that took subcutaneous readings extracutaneously, and despite being developed almost entirely by Asian engineers, it got the best data quality from fair-skinned Caucasians, and the worst from dark-skinned Africans, as it totally relied on the transmission of light through skin tissue. Claiming racism without understanding the problem is both unfair to the people involved, and an unwise undermining of one's own credibility for the times when people really are acting in bad faith. That may be in this case, but that may not be, and I don't like guilty-until-providen-innocent no matter whom it's leveled against (even a Federal agency of which I'm not the world's biggest fan, and a database the likes of which make me think we need a constitutional amendment to protect the privacy of the citizens).
  - mambodog 10y ago
    Even if it was no-one's intent that the system is racially biased, allowing such a system to continue operate after learning of such a bias is a racist act.
    - lgessler 10y ago
      I'm inclined to agree with you, but what would you say if it turned out that after much effort the engineering team was unable to make performance equal across all skin tones?
      - jacquesm 10y ago
        You mean that it would work better with darker skin?
apostacy 10y ago
Thank you for sharing that article. However, I don't see how the algorithm is itself racist. That is like, if I give you glasses to put on, and through those glasses, you observe racism. That doesn't mean that the glasses made racism appear. And I don't think attaching blinders so that you can't see the racism would be helpful, either. Word2vec is a lens that observes human output. And people saw things that they didn't want to see.
- Eyas 10y ago
  This is the thing about "systemic" racism. No one thing is racist. No need for racist intent. Just a combination of systems producing an outcome unfair to a particular group.
  - atomical 10y ago
    Could a disease be racist if it only targeted one ethnic group?
    - mambodog 10y ago
      If it was created by humans then that would constitute a racist act. This system was created by humans.
    - Eyas 10y ago
      No disease is. No algorithm is. No specific policy is. Systems can exhibit systemic racism. Other than hateful extremism coming from an individual, most forms of injustices are emergent properties.
      - sinxoveretothex 10y ago
        In a hypothetical world where evolution doesn't care about morals and therefore doesn't make sure to only create differences in morally insignificant features, how would you be able to tell what is "systemic discrimination" and what is the state of genetic differences across groups? Is the second also systemic discrimination? Can Joe the Down syndromer talk about systemic discrimination in having no chance at being the POTUS?
deleted 10y ago
[deleted]
IanDrake 10y ago
I call bullshit. It's physics, not racism. Darker colors absorb more light and reflect less. In a photo the contrast is difficult to bring out with darker subjects. I know this because I had a black dog and all her pictures came out like crap until she got old and turned salt and pepper. Some people just have to see racism everywhere. That's too bad because it takes meaning away from real racism.
- Eyas 10y ago
  I'm pointing out a statistical fact from the article that black people are more hurt by this technology. I'm not arguing for racist intent, but against accepting unfairness towards one group as acceptable. Dynamic range is of a photo is something controllable by humans building this technology. Contrast is a technical limitation and not a physical one. Lookup the articles on how early stage photography couldn't capture darker skin colors. Again, not racism, but related to incomes and advertising, but still had effects that disproportionally helped certain groups and harmed others.
maxxxxx 10y ago
My camera's eye and face detection are also pretty racist. Works well with white and asian people, not so much with black people. It must have been written by women though because it recognizes women much more than men.
- NoCoastCoder 10y ago
  Trouble dealing with beards maybe?

by Eyas 10y ago

abandonliberty 10y ago

That is a very interesting point regarding Word2vec, but it's inappropriate to call this example racism based on the evidence provided. Only 16% of the US population is Black, while 70% is white. If the error rates between Caucasians and Blacks were the same that would, in fact, be racist - or at the very least undemocratic. It's like asking that everything in the world be symmetric when only 10% of the world is left handed.

Eyas 10y ago
I don't see why being in a minority means accepting a higher chance of being confused by a criminal. This is how cycles of oppression propagate.
- eximius 10y ago
  Those aren't equivalent unless humans are out of the loop entirely, which they shouldn't be in any area of law.
  - Eyas 10y ago
    But then the humans can't be "color blind". They must explicitly go out of their way to undo the effects of the rest of the system by, E.g. Displaying extra care when dealing with a black suspect.
    - eximius 10y ago
      So, the usual, then?
- abandonliberty 10y ago
  Your chance of getting confused BY a criminal is probably more connected to your ability to comprehend and communicate. Perhaps you not understand the metaphor. Developing takes time and money to produce accuracy. You're advocating greater per-capita investment for an ethnic group. That's racial discrimination. You are advocating racism. It should come as no surprise that there's a difference. It is therefore suspect that they avoid mentioning the magnitude of the difference; it must not support their case otherwise they would highlight it. Critical reading, logic, and reason will help protect you from this type of manipulation and inadvertent racism.
anigbrowl 10y ago
Your conclusion doesn't follow from your premise, which leads me to wonder whether you understand your own argument.

rhcom2 10y ago

Reminds me of the soap dispenser that doesn't detect black skin. [1] 1: https://mic.com/articles/124899/the-reason-this-racist-soap-dispenser-doesn-t-work-on-black-skin#.yNg8U7lh4 https://mic.com/articles/124899/the-reason-this-racist-soap-...

chillacy 10y ago
Or cars and airbags being tested on male dummies for the longest time, resulting in women (who are shorter and weigh less) getting seriously injured/killed at higher rates in accidents: https://www.bloomberg.com/view/articles/2012-08-22/why-carmakers-always-insisted-on-male-crash-test-dummies https://www.bloomberg.com/view/articles/2012-08-22/why-carma... > In some cases, according to tests with female mannequins, small women were almost three times as likely as their average male counterparts to be seriously injured or killed. A study of actual crashes by the University of Virginia’s Center for Applied Biomechanics found that women wearing seatbelts were 47 percent more likely to be seriously injured than males in similar accidents.
bitwize 10y ago
There was a computer system featured once on Beyond 2000 that could recognize and mimic your facial expression. But it was developed by Japanese and tested only on Japanese people. The host had to wear a black wig to make it work.
gleenn 10y ago
The Xbox camera also had trouble tracking African Americans.

jplasmeier 10y ago

Facial unlocking programs have this problem too. Just search for "facial unlock racism" and there's pages of results from various sources and years. Software is subject to the same effects institutions of racism, sexism, etc. and is a good example of just how pervasive these institutions are.

sundvor 10y ago

In my naive mind, I can't help but think it might simply be a case of the contrast ratio being greater in lighter skinned people - therefore making the hit ratios higher. Greater contrast ratios will make facial features stand out more.

handedness 10y ago
Exactly my thinking. I worked with a company that developed on a medical device that took subcutaneous readings extracutaneously, and despite being developed almost entirely by Asian engineers, it got the best data quality from fair-skinned Caucasians, and the worst from dark-skinned Africans, as it totally relied on the transmission of light through skin tissue. Claiming racism without understanding the problem is both unfair to the people involved, and an unwise undermining of one's own credibility for the times when people really are acting in bad faith. That may be in this case, but that may not be, and I don't like guilty-until-providen-innocent no matter whom it's leveled against (even a Federal agency of which I'm not the world's biggest fan, and a database the likes of which make me think we need a constitutional amendment to protect the privacy of the citizens).
- mambodog 10y ago
  Even if it was no-one's intent that the system is racially biased, allowing such a system to continue operate after learning of such a bias is a racist act.
  - lgessler 10y ago
    I'm inclined to agree with you, but what would you say if it turned out that after much effort the engineering team was unable to make performance equal across all skin tones?
    - jacquesm 10y ago
      You mean that it would work better with darker skin?

apostacy 10y ago

Thank you for sharing that article. However, I don't see how the algorithm is itself racist. That is like, if I give you glasses to put on, and through those glasses, you observe racism. That doesn't mean that the glasses made racism appear. And I don't think attaching blinders so that you can't see the racism would be helpful, either. Word2vec is a lens that observes human output. And people saw things that they didn't want to see.

Eyas 10y ago
This is the thing about "systemic" racism. No one thing is racist. No need for racist intent. Just a combination of systems producing an outcome unfair to a particular group.
- atomical 10y ago
  Could a disease be racist if it only targeted one ethnic group?
  - mambodog 10y ago
    If it was created by humans then that would constitute a racist act. This system was created by humans.
  - Eyas 10y ago
    No disease is. No algorithm is. No specific policy is. Systems can exhibit systemic racism. Other than hateful extremism coming from an individual, most forms of injustices are emergent properties.
    - sinxoveretothex 10y ago
      In a hypothetical world where evolution doesn't care about morals and therefore doesn't make sure to only create differences in morally insignificant features, how would you be able to tell what is "systemic discrimination" and what is the state of genetic differences across groups? Is the second also systemic discrimination? Can Joe the Down syndromer talk about systemic discrimination in having no chance at being the POTUS?

deleted 10y ago

[deleted]

IanDrake 10y ago

I call bullshit. It's physics, not racism. Darker colors absorb more light and reflect less. In a photo the contrast is difficult to bring out with darker subjects. I know this because I had a black dog and all her pictures came out like crap until she got old and turned salt and pepper. Some people just have to see racism everywhere. That's too bad because it takes meaning away from real racism.

Eyas 10y ago
I'm pointing out a statistical fact from the article that black people are more hurt by this technology. I'm not arguing for racist intent, but against accepting unfairness towards one group as acceptable. Dynamic range is of a photo is something controllable by humans building this technology. Contrast is a technical limitation and not a physical one. Lookup the articles on how early stage photography couldn't capture darker skin colors. Again, not racism, but related to incomes and advertising, but still had effects that disproportionally helped certain groups and harmed others.

maxxxxx 10y ago

My camera's eye and face detection are also pretty racist. Works well with white and asian people, not so much with black people. It must have been written by women though because it recognizes women much more than men.

NoCoastCoder 10y ago
Trouble dealing with beards maybe?