3 ms·

ML systems are biased when data is biased. This face upsampling system makes everyone look white because the network was pretrained on FlickFaceHQ, which mainly

ML systems are biased when data is biased. This face upsampling system makes everyone look white because the network was pretrained on FlickFaceHQ, which mainly contains white people pics. Train the exact same system on a dataset from Senegal, and everyone will look African.

Afton 5y ago
https://twitter.com/timnitgebru/status/1274809417653866496?lang=en https://twitter.com/timnitgebru/status/1274809417653866496?l...
- Decker87 5y ago
  Can anyone ELI5 what the source of bias is outside of the biased training data?
  - I_Byte 5y ago
    I think that the Twitter thread is less about bias from datasets used to train ML apps and is more in reference to the application of biased ML apps that are supposed to identify criminals (which have been historically bad at identifying non-white individuals apart from each other). I feel as though the general message that the tweet threads author is trying to convey isn’t very clear. I wish they would clarify what they mean.
- throwjd764glug 5y ago
  > I’m sick of this framing. Tired of it. Many people have tried to explain, many scholars. Listen to us. You can’t just reduce harms caused by ML to dataset bias. Is pointing out the technical reason for a malfunction "reducing harms caused to dataset bias"? Actually, in this specific example, what form would the harm take place? Would the ML system upscale a surveillance photo of a black person into a white person, causing the white person to be wrongfully arrested? Does facial recognition working better on white faces mean white protesters are at greater risk of surveillance? I'm also sick of this framing.
- moralestapia 5y ago
  Funny how GP parroted the words of LeCunn, word by word. This is a big issue in Science, actually. One guy speaks and thousands follow. We are still in the dark ages.
  - 908B64B197 5y ago
    It's intentional. In a tweet he identified the issue with the model and how to fix it. Then he got mobbed for it. [0]. But he was, and still is, right. [0] https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter-amid-acrimonious-exchanges-on-ai-bias/ https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...
tveita 5y ago
> Train the exact same system on a dataset from Senegal, and everyone will look African. In other words it would still be broken. That's not much of a defense. I don't think there is dataset that would make the PULSE algorithm that produced that image deal well with skin color, the hill climbing it does is going to find it easier to take an averagish face and tweak the lighting.

Afton 5y ago

https://twitter.com/timnitgebru/status/1274809417653866496?lang=en https://twitter.com/timnitgebru/status/1274809417653866496?l...

Decker87 5y ago
Can anyone ELI5 what the source of bias is outside of the biased training data?
- I_Byte 5y ago
  I think that the Twitter thread is less about bias from datasets used to train ML apps and is more in reference to the application of biased ML apps that are supposed to identify criminals (which have been historically bad at identifying non-white individuals apart from each other). I feel as though the general message that the tweet threads author is trying to convey isn’t very clear. I wish they would clarify what they mean.
throwjd764glug 5y ago
> I’m sick of this framing. Tired of it. Many people have tried to explain, many scholars. Listen to us. You can’t just reduce harms caused by ML to dataset bias. Is pointing out the technical reason for a malfunction "reducing harms caused to dataset bias"? Actually, in this specific example, what form would the harm take place? Would the ML system upscale a surveillance photo of a black person into a white person, causing the white person to be wrongfully arrested? Does facial recognition working better on white faces mean white protesters are at greater risk of surveillance? I'm also sick of this framing.
moralestapia 5y ago
Funny how GP parroted the words of LeCunn, word by word. This is a big issue in Science, actually. One guy speaks and thousands follow. We are still in the dark ages.
- 908B64B197 5y ago
  It's intentional. In a tweet he identified the issue with the model and how to fix it. Then he got mobbed for it. [0]. But he was, and still is, right. [0] https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter-amid-acrimonious-exchanges-on-ai-bias/ https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...

tveita 5y ago

> Train the exact same system on a dataset from Senegal, and everyone will look African. In other words it would still be broken. That's not much of a defense. I don't think there is dataset that would make the PULSE algorithm that produced that image deal well with skin color, the hill climbing it does is going to find it easier to take an averagish face and tweak the lighting.