4 ms·

From the article: "Those disparities can sometimes be a matter of life or death: One recent study of the computer vision systems that enable self-driving cars

From the article: "Those disparities can sometimes be a matter of life or death: One recent study of the computer vision systems that enable self-driving cars to “see” the road shows they have a harder time detecting pedestrians with darker skin tones." Just think about the consequences of deploying such systems.

js2 8y ago
The story doesn’t cite the specific paper. But I think I submitted the paper last month: “Predictive Inequality in Object Detection” https://news.ycombinator.com/item?id=19379827 https://news.ycombinator.com/item?id=19379827 https://arxiv.org/abs/1902.11097 https://arxiv.org/abs/1902.11097 I’m not denying the conclusion of the paper, but it does have a lot of limitations. Also, Tesla notwithstanding, isn’t object detection primarily done via lidar?
jfk13 8y ago
I wonder if that's also true for human drivers. I certainly have a harder time detecting pedestrians wearing darker clothing, at least in some circumstances.
- skookumchuck 8y ago
  That's why road workers, cyclists, and joggers wear brightly colored reflective vests.
- munchbunny 8y ago
  There is also an important ethical difference between dark skin tones and dark clothing. I see the point you are making, but to the extent that an AI can do better than a human due to the physical aspect, for example, of dark skin reflecting less light at night, I think we should at least try.
tropo 8y ago
The computer vision systems could still beat real human vision systems. This could be by absolute numbers crashed into, or by having less bias than humans, or by both. It's not even a bias problem. It's a lighting, contrast, and camo problem. You could as well claim a bias between asphalt and concrete pavement, based on what skin tones are more visible with the pavement as a background.
- externalreality 8y ago
  > The computer vision systems could still beat real human vision systems. For pattern recognition, No, the computer vision system can't best a human.
  - tropo 8y ago
    I'm not saying it does today. I'm saying it could do so, while still having bias. The existence of bias doesn't make the computer worse than humans.
  - deogeo 8y ago
    But the entire self-driving system can. It is never tired, or distracted, has far fewer blind spots, and possibly LIDAR and IR to help it.
    - scottie_m 8y ago
      It can’t today, and until it proves to do so making claims about future tech in relation to present policy is a bad idea.
- int_19h 8y ago
  Suppose the system does beat real human vision by absolute numbers , but it turns out that it is objectively worse for someone who is not white, and the higher average is because the majority of people in that society are white. Is that acceptable?
  - deleted 8y ago
    [deleted]
colllectorof 8y ago
So maybe we should not be using opaque ML algorithms for life-and-death situations to begin with? Instead of, you know, hoping that whomever is screwed by them has an active advocacy group in places like MIT.
- mikeash 8y ago
  Using NL for such things hasn’t been working too great, so it seems worthwhile to try for something better.
  - hprotagonist 8y ago
    such as?
    - mikeash 8y ago
      Are you asking for examples where NL hasn’t done well or for potential ways to do better?
      - hprotagonist 8y ago
        the latter. It’s relatively easy to say “stop doing X”; it’s harder to say “...and have you considered Y instead?”
        mikeash 8y ago
        I’d hope it was obvious from context that ML was the potential way to do better here.
        hprotagonist 8y ago
        I read you as saying that it was the thing doing poorly and that some superior object detection method was required. What that approach could be is totally unobvious to me.
sonnyblarney 8y ago
"Just think about the consequences of deploying such systems." Those systems are already widely deployed - they're called 'humans'. All vision systems are going to be more likely to confuse like colours. This problem is quite a bit different from the 'mugshot' problem, or a problem wherein there was a bias or lack of training data for certain samples. The author I think conflated a lot of the issues and just boiled it down to 'evil AI' which I don't think is the right thing to do. Issues of ethnic orientation are quite a bit different from systems that have trouble literally due to the colour of something.
maskymask 8y ago
I saw a talk by one of the authors of the pedestrian detection study that I think they are talking about, and their results did not seem support the scary racist self-driving car hypothesis that the study clearly implies (only marginal/statistically insignificant effects, with very little controls for confounding factors (I believe they controlled for bounding box size or something, but very little else)). With that said, fairness in ML/AI is a real problem, and some people are doing some really good/important work in this area. I am not familiar with Buolamwini's work, but I'm much more inclined to believe disparities in facial recognition than pedestrian detection where very little skin is visible and almost everything you're seeing is clothing.
deleted 8y ago
[deleted]

js2 8y ago

The story doesn’t cite the specific paper. But I think I submitted the paper last month: “Predictive Inequality in Object Detection” https://news.ycombinator.com/item?id=19379827 https://news.ycombinator.com/item?id=19379827 https://arxiv.org/abs/1902.11097 https://arxiv.org/abs/1902.11097 I’m not denying the conclusion of the paper, but it does have a lot of limitations. Also, Tesla notwithstanding, isn’t object detection primarily done via lidar?

jfk13 8y ago

I wonder if that's also true for human drivers. I certainly have a harder time detecting pedestrians wearing darker clothing, at least in some circumstances.

skookumchuck 8y ago
That's why road workers, cyclists, and joggers wear brightly colored reflective vests.
munchbunny 8y ago
There is also an important ethical difference between dark skin tones and dark clothing. I see the point you are making, but to the extent that an AI can do better than a human due to the physical aspect, for example, of dark skin reflecting less light at night, I think we should at least try.

tropo 8y ago

The computer vision systems could still beat real human vision systems. This could be by absolute numbers crashed into, or by having less bias than humans, or by both. It's not even a bias problem. It's a lighting, contrast, and camo problem. You could as well claim a bias between asphalt and concrete pavement, based on what skin tones are more visible with the pavement as a background.

externalreality 8y ago
> The computer vision systems could still beat real human vision systems. For pattern recognition, No, the computer vision system can't best a human.
- tropo 8y ago
  I'm not saying it does today. I'm saying it could do so, while still having bias. The existence of bias doesn't make the computer worse than humans.
- deogeo 8y ago
  But the entire self-driving system can. It is never tired, or distracted, has far fewer blind spots, and possibly LIDAR and IR to help it.
  - scottie_m 8y ago
    It can’t today, and until it proves to do so making claims about future tech in relation to present policy is a bad idea.
int_19h 8y ago
Suppose the system does beat real human vision by absolute numbers , but it turns out that it is objectively worse for someone who is not white, and the higher average is because the majority of people in that society are white. Is that acceptable?
- deleted 8y ago
  [deleted]

colllectorof 8y ago

So maybe we should not be using opaque ML algorithms for life-and-death situations to begin with? Instead of, you know, hoping that whomever is screwed by them has an active advocacy group in places like MIT.

mikeash 8y ago
Using NL for such things hasn’t been working too great, so it seems worthwhile to try for something better.
- hprotagonist 8y ago
  such as?
  - mikeash 8y ago
    Are you asking for examples where NL hasn’t done well or for potential ways to do better?
    - hprotagonist 8y ago
      the latter. It’s relatively easy to say “stop doing X”; it’s harder to say “...and have you considered Y instead?”
      - mikeash 8y ago
        I’d hope it was obvious from context that ML was the potential way to do better here.
        hprotagonist 8y ago
        I read you as saying that it was the thing doing poorly and that some superior object detection method was required. What that approach could be is totally unobvious to me.

sonnyblarney 8y ago

"Just think about the consequences of deploying such systems." Those systems are already widely deployed - they're called 'humans'. All vision systems are going to be more likely to confuse like colours. This problem is quite a bit different from the 'mugshot' problem, or a problem wherein there was a bias or lack of training data for certain samples. The author I think conflated a lot of the issues and just boiled it down to 'evil AI' which I don't think is the right thing to do. Issues of ethnic orientation are quite a bit different from systems that have trouble literally due to the colour of something.

maskymask 8y ago

I saw a talk by one of the authors of the pedestrian detection study that I think they are talking about, and their results did not seem support the scary racist self-driving car hypothesis that the study clearly implies (only marginal/statistically insignificant effects, with very little controls for confounding factors (I believe they controlled for bounding box size or something, but very little else)). With that said, fairness in ML/AI is a real problem, and some people are doing some really good/important work in this area. I am not familiar with Buolamwini's work, but I'm much more inclined to believe disparities in facial recognition than pedestrian detection where very little skin is visible and almost everything you're seeing is clothing.

deleted 8y ago

[deleted]