6 ms·

Let's take the classic example of a "racist algorithm" Image recognition using neural nets and examine what's going on. Let's say we train the NN using equal n

Let's take the classic example of a "racist algorithm" Image recognition using neural nets and examine what's going on. Let's say we train the NN using equal number of human faces of all races, and animals faces. Let's say the trained neural net made some mistakes, including a few cases where black faces are recognized as gorilla faces. And this doesn't happen at all or as likely with white faces. The results are horrible, right? And people immediately start to point fingers to the training data and the algorithms, stating the training data is racially biased and/or the training algorithms or even the use of neural nets are racially biased. But is it really so? It's known that in order to take a black face photo with the same degree of details, the lighting condition and camera settings need to be adjusted. This is an effect purely due to physics. In other words, it could well be by nature, that recognizing black faces is harder than pale faces under the normal camera and scene settings. This is why you have night mode on your phones. It is just harder to take clear photos when less light gets into the camera. And this requires the camera and photography settings to be adjusted. So, the unwanted results here are still due to the input data. But neither the input data, nor the algorithms contain any racial bias towards the black people. The results might be merely due to the difference between dark faces and pale faces under the natural law of physics. These are unwanted results due to our social norms, but they are NOT racially biased or racist, because there is no such bias introduced or inherent during any part of the process. We could and should correct such unwanted results by introducing adjustments to the input data, like improving dark face photography and camera sensitivity. But we can't just label the input data, the algorithms and the people who designed these algorithms as "racist" or "racially-biased". There is zero racial bias that is man-made here. The race just coincide with the side-effect of photography. Likewise, there will be cases where the reverse is true, like white faces get unwanted results instead of dark faces. So, while we work towards improving the data quality and the algorithms, we must stop this trend of labeling or calling people and algorithms racists.

deleted 5y ago
[deleted]
MichaelGroves 5y ago
> So, the unwanted results here are still due to the input data. But neither the input data, nor the algorithms contain any racial bias towards the black people. The results might be merely due to the difference between dark faces and pale faces under the natural law of physics. These are unwanted results due to our social norms, but they are NOT racially biased or racist, because there is no such bias introduced or inherent during any part of the process. In this, and the reasoning above it, I think you are correct. Assuming we are right about this, what is the next step? You say the next step is to improve data collection, e.g. by creating better cameras. That seems a fine proposal to me, I support that, but I think there is more that might also be done. For instance, the use of ML models could be restricted or regulated in at least some contexts, until the problems with data collection are rectified. For instance, we could ban the police from using facial recognition models until the problems with data collection are solved. The bias is a side-effect of photography, not something intrinsic to the facial recognition algorithm, but restricting the use of that algorithm might nonetheless be a valid response to this circumstance.
jbattle 5y ago
> In other words, it could well be by nature, that recognizing black faces is harder than pale faces under the normal camera and scene settings. This might be the crux. Why are pale-face-recognizing settings the "normal" settings? Why aren't the cameras designed and tuned to recognize darker skinned faces by default? Cameras are designed and tuned by people - this is not a matter of fundamental physics having a preference.
- deleted 5y ago
  [deleted]
- dthul 5y ago
  It's not as easy as just retuning camera settings. Due to physical limitations (at least with our current state of technology) camera sensors have a very limited dynamic range compared to the human eye. Increasing the exposure to better image darker surfaces will overexpose the rest of the image. We can be hopeful though that this will become less of an issue in the future due to camera technology advancements (like HDR exposure stacks).
SpicyLemonZest 5y ago
The source article seems to agree with you on this point, and does not call any person or algorithm "racist". I think they understand the term "bias" to mean simply "things we might want to introduce adjustments for".
bsanr2 5y ago
>Let's say we train the NN using equal number of human faces of all races, and animals faces. You're describing something that happened, and the data sets very much did NOT represent a smooth spectrum of skin tones. It was heavily weighted towards light skin. The premise is incongruent with reality. Your analysis is fundamentally flawed.

deleted 5y ago

[deleted]

MichaelGroves 5y ago

> So, the unwanted results here are still due to the input data. But neither the input data, nor the algorithms contain any racial bias towards the black people. The results might be merely due to the difference between dark faces and pale faces under the natural law of physics. These are unwanted results due to our social norms, but they are NOT racially biased or racist, because there is no such bias introduced or inherent during any part of the process. In this, and the reasoning above it, I think you are correct. Assuming we are right about this, what is the next step? You say the next step is to improve data collection, e.g. by creating better cameras. That seems a fine proposal to me, I support that, but I think there is more that might also be done. For instance, the use of ML models could be restricted or regulated in at least some contexts, until the problems with data collection are rectified. For instance, we could ban the police from using facial recognition models until the problems with data collection are solved. The bias is a side-effect of photography, not something intrinsic to the facial recognition algorithm, but restricting the use of that algorithm might nonetheless be a valid response to this circumstance.

jbattle 5y ago

> In other words, it could well be by nature, that recognizing black faces is harder than pale faces under the normal camera and scene settings. This might be the crux. Why are pale-face-recognizing settings the "normal" settings? Why aren't the cameras designed and tuned to recognize darker skinned faces by default? Cameras are designed and tuned by people - this is not a matter of fundamental physics having a preference.

deleted 5y ago
[deleted]
dthul 5y ago
It's not as easy as just retuning camera settings. Due to physical limitations (at least with our current state of technology) camera sensors have a very limited dynamic range compared to the human eye. Increasing the exposure to better image darker surfaces will overexpose the rest of the image. We can be hopeful though that this will become less of an issue in the future due to camera technology advancements (like HDR exposure stacks).

SpicyLemonZest 5y ago

The source article seems to agree with you on this point, and does not call any person or algorithm "racist". I think they understand the term "bias" to mean simply "things we might want to introduce adjustments for".

bsanr2 5y ago

>Let's say we train the NN using equal number of human faces of all races, and animals faces. You're describing something that happened, and the data sets very much did NOT represent a smooth spectrum of skin tones. It was heavily weighted towards light skin. The premise is incongruent with reality. Your analysis is fundamentally flawed.