5 ms·

Self-driving cars fail to detect dark-skinned pedestrians

23 points by pm24601 8y ago

NebeisHa 8y ago
Sounds like a physical issue, about how dark surfaces reflect less light and the car can't analyze that information. They'll find a way to fix this eventually.
waste_monk 8y ago
I do not like the conclusion of the article. Increasing the diversity of the team members might be good for reducing bias, but the fact remains that darker skinned people are harder for machine vision systems to work with due to the lack of contrast. Like that fuss a few years ago with the laptops (I want to say it was Toshiba but I'm not certain) not being able to use facial recognition to unlock the OS. Also, this seems to be only vision systems - you'd hope LIDAR would be able to detect a pedestrian regardless of skin color.
- Ace17 8y ago
  > the fact remains that darker skinned people are harder for machine vision systems to work with due to the lack of contrast. Is it also harder for human drivers?
  - ASalazarMX 8y ago
    Depending on clothing and light conditions, yes. We don't have LIDARs, though.
- samd1986 8y ago
  I would a real surprise if models with LIDAR inputs can detect your skin color. I also do not see what team diversity should do. Adding such a bias actually on purpose is way harder than one would think (and not the issue), and simply making a better model should be the goal of any team anyway. In the study they do say that at night the dark skinned people could actually be detected better (according to their very small validation set). Something nobody would expected. So I'd really be worried if their model makes it into production. They still have some issues at least with sample size. The thing is these types of models are really tricky to get right. When they are wrong/biased it is just because it is difficult to get right. I think it is stupid to frame this problem as a racial issue.
- malms 8y ago
  And probably also people with black coats at night / white coats in front of white walls ...
medecau 8y ago
For those interested in learning more about these problems there are two interesting book to read. "Technicaly Wrong" and "Weapons of Math Destruction"
Ace17 8y ago
> The report [...] hasn’t yet been peer-reviewed. It didn’t test any object-detection models actually being used by self-driving cars, nor did it leverage any training datasets actually being used by autonomous vehicle manufacturers.
Jyaif 8y ago
How information gets propagated in 2019: 1. The study talks about how some vision-based object detection systems don't work as well on black people. 2. The articles makes the leap to self driving cars, ignoring that self driving cars uses Lidar and probably custom made algorithms. To their credit they do say that it's a "potential risk". 3. Everybody shares (including this HN submission) or rewrite the article dropping the "potential" part.
alkibiades 8y ago
how do idiots blame bias for everything? if you have a dark room of course it’ll be harder to detect a dark skin person! the same with a couch that’s black instead of white.
- senectus1 8y ago
  its an issue for ocular recognition specifically. I'd say that is worth noting! If a automated car is better at detecting threats on the road than humans then there is little point to using them.
samd1986 8y ago
For all those not bothering to check the study. It actually says "At night the model was BETTER at DETECTING DARK skinned people than LIGHT". Questionable result but still in there.
panarky 8y ago
Step 1 - find an object detection model that's not used in self-driving cars Step 2 - show that the model detects dark-skinned pedestrians 5% less often than light-skinned pedestrians Step 3 - write a clickbait title implying that self-driving cars will run down dark-skinned pedestrians with precisely zero evidence Step 4 - post the article on HN with an even more inflammatory title than the actual article
- pm24601 8y ago
  FYI - I had to trim title because the original title was too long. Take this up with HN
jacknews 8y ago
surely this is simple physics (white surfaces reflect more light), rather than algorithmic bias

23 points by pm24601 8y ago

Sounds like a physical issue, about how dark surfaces reflect less light and the car can't analyze that information. They'll find a way to fix this eventually.

waste_monk 8y ago

I do not like the conclusion of the article. Increasing the diversity of the team members might be good for reducing bias, but the fact remains that darker skinned people are harder for machine vision systems to work with due to the lack of contrast. Like that fuss a few years ago with the laptops (I want to say it was Toshiba but I'm not certain) not being able to use facial recognition to unlock the OS. Also, this seems to be only vision systems - you'd hope LIDAR would be able to detect a pedestrian regardless of skin color.

Ace17 8y ago
> the fact remains that darker skinned people are harder for machine vision systems to work with due to the lack of contrast. Is it also harder for human drivers?
- ASalazarMX 8y ago
  Depending on clothing and light conditions, yes. We don't have LIDARs, though.
samd1986 8y ago
I would a real surprise if models with LIDAR inputs can detect your skin color. I also do not see what team diversity should do. Adding such a bias actually on purpose is way harder than one would think (and not the issue), and simply making a better model should be the goal of any team anyway. In the study they do say that at night the dark skinned people could actually be detected better (according to their very small validation set). Something nobody would expected. So I'd really be worried if their model makes it into production. They still have some issues at least with sample size. The thing is these types of models are really tricky to get right. When they are wrong/biased it is just because it is difficult to get right. I think it is stupid to frame this problem as a racial issue.
malms 8y ago
And probably also people with black coats at night / white coats in front of white walls ...

medecau 8y ago

For those interested in learning more about these problems there are two interesting book to read. "Technicaly Wrong" and "Weapons of Math Destruction"

Ace17 8y ago

> The report [...] hasn’t yet been peer-reviewed. It didn’t test any object-detection models actually being used by self-driving cars, nor did it leverage any training datasets actually being used by autonomous vehicle manufacturers.

Jyaif 8y ago

How information gets propagated in 2019: 1. The study talks about how some vision-based object detection systems don't work as well on black people. 2. The articles makes the leap to self driving cars, ignoring that self driving cars uses Lidar and probably custom made algorithms. To their credit they do say that it's a "potential risk". 3. Everybody shares (including this HN submission) or rewrite the article dropping the "potential" part.

alkibiades 8y ago

how do idiots blame bias for everything? if you have a dark room of course it’ll be harder to detect a dark skin person! the same with a couch that’s black instead of white.

senectus1 8y ago
its an issue for ocular recognition specifically. I'd say that is worth noting! If a automated car is better at detecting threats on the road than humans then there is little point to using them.

samd1986 8y ago

For all those not bothering to check the study. It actually says "At night the model was BETTER at DETECTING DARK skinned people than LIGHT". Questionable result but still in there.

panarky 8y ago

Step 1 - find an object detection model that's not used in self-driving cars Step 2 - show that the model detects dark-skinned pedestrians 5% less often than light-skinned pedestrians Step 3 - write a clickbait title implying that self-driving cars will run down dark-skinned pedestrians with precisely zero evidence Step 4 - post the article on HN with an even more inflammatory title than the actual article

pm24601 8y ago
FYI - I had to trim title because the original title was too long. Take this up with HN

jacknews 8y ago

surely this is simple physics (white surfaces reflect more light), rather than algorithmic bias