6 ms·

“Aren't people primates?” Words have meaning on multiple levels. On a technical level, yes, all humans are primates, but if the algorithm were following this

“Aren't people primates?” Words have meaning on multiple levels. On a technical level, yes, all humans are primates, but if the algorithm were following this rule, it would tag all humans in all photos as primates, but it didn’t do that. It only tagged black people. That takes us to a social meaning level. In this context, you have to consider the long, racists history of dehumanizing black people. They have often been called “monkey” and “ape” with the intent to degrade and to segregate them from “people”. In that light, and the fact that the label was not applied in all cases where appropriate, it is hard to see it as anything but racist. Probably not deliberately but if the training material contained racist material, it could bias the result. Even if the training material was inadvertently unbalanced and contained very few black faces, the algorithm would have less experience with black faces and would not know how to categorize them as well as white faces. This is analogous to people who grow up and live largely isolated from close associates with those of other races. Their experiences are filled with varied samples of interactions with people of their own race, but are poor in samples with other races. In the case where we have less experience, we tend to fall back on broad stereotypes and our reactions show strong racial bias.

Y_Y 5y ago
I've dealt with a lot of badly labelled datasets, but I can't say I ever saw one maliciously labelled by racists. There are other more subtle problems that may contribute. As an amateur photographer I've been told that most cameras are calibrated for light skin. If you aren't careful with your setup you'll get bad definition in shadows and relatively very dark areas. If you have badly calibrated/exposed photos then very dark faces, whether of sub-Saharan people or gorillas, may be poorly distinguished. I bet there are loads of unfair, hard-to-catch issues like that. By the way I don't mean to say that you're wrong, but just to offer a Hanlon's Razor-type counterpoint as it occurred to me.
- dagmx 5y ago
  This is usually a good example of societal bias compounding. It's possible that nobody directly involved in this was actively racist, but we're building on centuries of racism in every facet. So a little bit of bias in each technology and data set involved, not to mention the engineers and their testing, just adds up
  - salawat 5y ago
    >but we're building on centuries of racism in every facet. Nooo. We're building on techniques that have worked well enough up til now, but people don't bother to dig in a d come to terms with what's actually going on. Not everything is racist. It's more a tyranny of 'good enough' and convenience.
    - dagmx 5y ago
      So you're saying there's no bias in the data sets , or in society? Do you think that someone who thinks there's no bias in their dataset would then dig in to see what's going on? Do you think people are going to recognize the negative biases in the data if they don't face those issues themselves? Bigotry is part of society, because it's existed for so long and still does. It doesn't mean that everyone is actively racist. It just means you can't just say there's no bigotry affecting things, and in saying there isn't, you're now blinding yourself to the possibility of negative biases.
    - dv_dt 5y ago
      Once we know about the the systemic bias, even if accumulated unintentionally - then what do we do about it. I think we are obligated morally and technically to do more after that point. A similar situation exists with accommodation of people with disabilities. Once you know a building architecture can shut out people by design decisions- at some point we worked through at least some basic level of agreement on principle that accommodation is required.
- Tagbert 5y ago
  That was my point. If the training data contained mostly white faces and few black faces, the AI would have fewer cases to base it’s weightings when trying to analyze a photo with black faces.
- nzeribe 5y ago
  I'm sorry but no. I don't believe this at all. What are the chances of say, you finding a dataset maliciously labelled by racists and how would you uncover that? Would you ask every software engineer how he felt about Black Lives Matter as he tagged images? It's more likely that there is malicious intent in a company where 2% of software engineers are African-American than not.

Y_Y 5y ago

I've dealt with a lot of badly labelled datasets, but I can't say I ever saw one maliciously labelled by racists. There are other more subtle problems that may contribute. As an amateur photographer I've been told that most cameras are calibrated for light skin. If you aren't careful with your setup you'll get bad definition in shadows and relatively very dark areas. If you have badly calibrated/exposed photos then very dark faces, whether of sub-Saharan people or gorillas, may be poorly distinguished. I bet there are loads of unfair, hard-to-catch issues like that. By the way I don't mean to say that you're wrong, but just to offer a Hanlon's Razor-type counterpoint as it occurred to me.

dagmx 5y ago
This is usually a good example of societal bias compounding. It's possible that nobody directly involved in this was actively racist, but we're building on centuries of racism in every facet. So a little bit of bias in each technology and data set involved, not to mention the engineers and their testing, just adds up
- salawat 5y ago
  >but we're building on centuries of racism in every facet. Nooo. We're building on techniques that have worked well enough up til now, but people don't bother to dig in a d come to terms with what's actually going on. Not everything is racist. It's more a tyranny of 'good enough' and convenience.
  - dagmx 5y ago
    So you're saying there's no bias in the data sets , or in society? Do you think that someone who thinks there's no bias in their dataset would then dig in to see what's going on? Do you think people are going to recognize the negative biases in the data if they don't face those issues themselves? Bigotry is part of society, because it's existed for so long and still does. It doesn't mean that everyone is actively racist. It just means you can't just say there's no bigotry affecting things, and in saying there isn't, you're now blinding yourself to the possibility of negative biases.
  - dv_dt 5y ago
    Once we know about the the systemic bias, even if accumulated unintentionally - then what do we do about it. I think we are obligated morally and technically to do more after that point. A similar situation exists with accommodation of people with disabilities. Once you know a building architecture can shut out people by design decisions- at some point we worked through at least some basic level of agreement on principle that accommodation is required.
Tagbert 5y ago
That was my point. If the training data contained mostly white faces and few black faces, the AI would have fewer cases to base it’s weightings when trying to analyze a photo with black faces.
nzeribe 5y ago
I'm sorry but no. I don't believe this at all. What are the chances of say, you finding a dataset maliciously labelled by racists and how would you uncover that? Would you ask every software engineer how he felt about Black Lives Matter as he tagged images? It's more likely that there is malicious intent in a company where 2% of software engineers are African-American than not.