8 ms·
Self-driving cars less likely to detect children and people of color
- im3w1l 3y agoThe researchers supposedly used similar but not quite the same approach as Tesla, and claimed it worked worse for people of color. That makes sense since optical recognition is harder for dark skinned people. However, the article leads with a picture of a Cruise car, which use lidar technology. Those should afaik recognize people with the same accuracy regardless of skin color.
- joe__f 3y agoAlso, children are small
- seanmcdirmid 3y agoI thought Waymo and Cruise are both using optical and LiDAR (using fancy sensor fusion algorithms to make those work safely together), so this would impact them as well? LiDAR alone isn’t going to be able to tell if something is a pedestrian, just that something is there, optical is still needed to determine that the thing is a person.
- GaggiX 3y ago>This is due to bias in open-source AI, on which self-driving cars rely, researchers say. Or they are simply less visible.
- gcanyon 3y ago> This is due to bias in open-source AI, on which self-driving cars rely, researchers say. The second line in the article
- hardware2win 3y agoHow do they know?
- throw__away7391 3y agoBecause that is the answer to every single question.
- DennisP 3y agoYes but by "bias" they just mean the miss rate is different. Digging into the study, they talk about visibility and contrast, and checked whether the problem is worse at night (which it is). They don't appear to be suggesting that the algorithms are prejudiced against darker-skinned people.
- cj 3y agoWhat's "less visible" to the human eye shouldn't be used to lower our expectations of what can be visible to a machine.
- GaggiX 3y agoWell if it's using optical recognition then lower contrast or smaller object will always make the job more difficult then a bigger object with more contrast; then the reason why the model is better at recognizing white adult people. Even if the car is never going to strike a child or a black person the model recognition would still be better with them.
- willcipriano 3y agoDon't let physics get in the way of your ideals.
- elcano 3y agoYou mean, to all possible sensors and spectrum signals in the arsenal?
- lsaferite 3y agoPretty sure they are discussing 3-band cameras between 400nm and 700nm (roughly). If we start adding multi-spectral or hyper-spectral cameras to the mix then you'd have a lot more information to use, but then you'd have a lost more information to process.
- Lewton 3y agothey're testing 8 different detection algorithms > The detection systems were 19.67% more likely to detect adults than children, and 7.52% more likely to detect people with lighter skin tones than people with darker skin tones, according to the study. while they all had a harder time with adults vs children, that 7.52% is gotten by averaging 2 algorithms that performed abysmally, with 6 that had no statistically significant differences https://arxiv.org/pdf/2308.02935.pdf https://arxiv.org/pdf/2308.02935.pdf table 6
- strken 3y agoThe two with significantly worse performance were RetinaNet and YOLOX. I don't really know anything about the field, but it's interesting they're both single stage performant models, while the slower but lower miss-rate RCNN variants are two-stage. It's interesting that the pedestrian-specific models are all worse than the general models at detecting people! The conclusion is kind of weird: apparently their "findings reveal significant bias in the current pedestrian detectors" despite the bias being almost entirely within the single-pass general object detectors. And where it's statistically significant in the other models, the miss rate is low in both cases, and the effect is reversed! (Dry-weather Cascade-RCNN does better on dark-skin than light-skin, among others.)
- yorwba 3y agoI think you misunderstood table 6. All algorithms show significant differences in miss rate for children, two show significant differences based on gender, and four others based on skin color. The four that showed no statistically significant difference between light and dark skin had very high miss rates overall. Of the other four, two are much worse for dark skin, and two are slightly better. Those last two are also best at detecting children, but 28% miss rate is still a bit too high for my taste.
- Lewton 3y agoYeah I missed the two statistically significant algorithms that favor darker skin since they have smaller percentage differences than the ones they didn't mark as statistically significant (but I guess that's because of how it relates to the overall miss rate) RE: 28% miss rate, I think this is meaningless as it's looking at single images/data points, while self driving cars get a continuous stream of data
- zirgs 3y agoHow do they work in winter then? You can't see much skin if someone is wearing a winter coat. Right - self driving cars are a solution for Silicon Valley only only so they don't even bother testing those cars elsewhere.
- mrkeen 3y ago> A team of researchers in the UK and China tested how well eight popular pedestrian detectors worked depending on a person's race, gender, and age. - edit - Sorry, I read the article too quickly and assumed it was talking about the countries UK and China. Perhaps they only bothered testing the cards in UK, Silicon Valley and China, Silicon Valley.
- mrweasel 3y agoThe skin color is going to be the least of your issues in the winter time. How are the cars going to "see" the road under 10cm of snow? Granted humans shouldn't really be driving in those conditions either, but we do and mostly successfully, to avoid sleeping on the side of the road in -10C. What is it that makes it so hard for all these algorithms to work on people with darker skin? This has been an issue for more than ten years, surely someone has started adding various skin colors into the training data. Is it a case of lack of training material, or is it just faster to focus on one skin type?
- snakeyjake 3y agoMy employer makes a multi-band synthetic aperture radar that penetrate snow and is high-resolution enough to "see" painted road markings and reflectors beneath a layer of snow. It is nowhere near small or cheap enough for self-driving car applications, but will be one day. Another challenge is affordable real-time processing of the data. Churning through 3,200MB/s of phase-history data is expensive but again that will solve itself given time.
- mrweasel 3y agoThat is pretty neat, but what about the road with no markings? None of the road leading into the town where I live have road marking or reflectors or are you able to target the reflectors on posts by the side of the road as well? I mean see through the snow on those.
- gcanyon 3y agoAnyone know how likely it is that this is the result of imbalanced training data? You have fewer dark-skinned people and children in the training data, you end up with a model less-skilled at detecting those people.
- causi 3y agoYou'd have to compare the machine's performance to real human performance. I suspect humans also have an uneven detection rate between those categories.
- boomboomsubban 3y agoIt's strange that the paper doesn't seem to include any of the actual data, but it is available on their github page https://github.com/FairnessResearch/Fairness-Testing-of-Autonomous-Driving-Systems https://github.com/FairnessResearch/Fairness-Testing-of-Auto... From what I can see, a couple of the detectors used really seem shit overall, making the combined data of questionable value.
- Eddy_Viscosity2 3y agoIt won't be long till they won't just be detecting people, by identifying specifically which people they are. Think of the data! All those cars logging time/location of all those people. (which, of course, will only be used for good and the occasional targeted ad)
- gizajob 3y agoUrgh
- blibble 3y ago[flagged]
- candiddevmike 3y agoI think what you're trying to say is cars will start identifying folks and logging their location. Cell tracking/triangulation data is already available from cell companies. Self driving cars aren't going to make things worse.
- helmette 3y agoCarrying a mobile phone is optional.
- ben_w 3y agoAnd visual detection more precise. You can tell not only if someone is usually within a few tens of metres as you can with cellphone data, but exactly where they are, what they're doing, and who/what they're looking at.
- Eddy_Viscosity2 3y agoOptimistic to think that having more ways to track people won't make things worse.
- solarkraft 3y agoBuddy, you're commenting on an article showing that cars aren't even that good at identifying people as people.
- haunter 3y agoThe abstract sounds a bit different than the (rage baiting) article >bias towards dark-skin pedestrians increases significantly under scenarios of low contrast and low brightness https://arxiv.org/pdf/2308.02935.pdf https://arxiv.org/pdf/2308.02935.pdf
- jl6 3y ago> While the study did not use the exact software companies like Tesla use to power self-driving cars because they are confidential, the software systems used for the study are based on the same open-source AI those companies use, according to Zhang. I was under the impression that commercial self-driving software was deeply proprietary and confidential, and there is no way to know that this study will generalize if run on state of the art detectors. Tesla and Cruise are name-checked in the article - how do we know this isn’t a problem they have worked extensively on and made great improvements to, relative to the open source components? Feels like a case of outrage-for-clicks.
- candiddevmike 3y agoThe point of the article, to me, is computer vision will never be enough. These are machines and need to be augmented with radar and other object detection methods.
- AdamN 3y agoIsn't radar and object detection standard for these types of systems?
- mrkeen 3y agoThey certainly should be. Tesla stopped selling cars with radar in 2021: https://www.carscoops.com/2023/05/tesla-is-disabling-radar-sensors-from-customer-cars-during-routine-service-checks/ https://www.carscoops.com/2023/05/tesla-is-disabling-radar-s... Elon Musk Overruled Tesla Engineers Who Said Removing Radar Would Be Problematic: https://insideevs.com/news/658439/elon-musk-overruled-tesla-autopilot-engineers-radar-removal/ https://insideevs.com/news/658439/elon-musk-overruled-tesla-...
- candiddevmike 3y agoTesla does not: https://www.tesla.com/support/transitioning-tesla-vision#:~:text=In%202021%2C%20we%20began%20our,our%20camera%2Dbased%20Autopilot%20system https://www.tesla.com/support/transitioning-tesla-vision#:~:....
- Andrew018 3y ago[flagged]
- HunterWare 3y agoIn other news, it turns out that detecting smaller and lower contrast objects is harding with optical sensors. Almost, you know, like how it is with real people.
- hermannj314 3y ago"A new technology reduces mortality risk for all people, but has slightly better outcomes for white adults." Conclusion - we call on lawmakers to make this technology illegal. We prefer more people die at equal rates more than we prefer less people to die at unequal rates. I am not sure I agree with the ethics that underlies this way of seeing the world.
- candiddevmike 3y agoIt should be illegal, the current systems will never deliver without additional object detection methods (radar, lidar, etc). Get this shit recalled and back to the drawing board.
- luma 3y agoDo you have lidar and radar when you drive? Should it be illegal for you to drive without those things?
- notamy 3y agoWhat is it with people on this website (not specifically the user I’m replying to) and going with "the computer vaguely does the thing the human does, literally what's the difference???" (or similar) as an argument/gotcha/etc. around ML/AI/self-driving/...? It’s painfully obvious that the computer and the human are different systems that process information differently and therefore likely need different sets of inputs for optimal function.
- matwood 3y agoI agree that they are different, but if I read the comment you responded to charitably, it's pointing out that the bar often seems unreasonably high for self driving. Human drivers are terrible leading to ~100 deaths/day in the US. How much better does self-driving need to be in order to make it worthwhile? Does it have to be perfect?
- 3y ago
- steveBK123 3y agoTo be fair, human drivers would likely have the same problem given seating position in cars & certain lighting conditions..
- skinkestek 3y agoCan confirm to some degree. I had one dark skinned kid in dark clothes casually crossing the road in front of me during the dark months here were I live. I didn't have to slam the brakes or anything because it was a bit ahead of me, bit it was scary because of how hard it was to detect.
- amluto 3y agoTo be fair, in cool or cold weather, a pedestrian can easily have little or so visible skin — long sleeves, hair, and possibly a hat of any sort can hide skin very well. Pedestrians (and cyclists) should wear bright clothing and/or retroreflectors at night! (Ideally both. Retroreflectors are mostly useless if a car has its lights off.)
- skinkestek 3y agoAround here you get a fine if you drive with the lights off, even in the summer.
- steveBK123 3y agoTaking your life in your hands (biking in dark clothing, especially at night) because someone is going to get a fine for having their lights off after they kill you is not a good risk calculation.
- lemper 3y agoas much as I dislike "AI" and its adjacent topics in HN, I think this can be solved with the companies which have the stake to get data from Asia and African nations. I don't know, pay someone group of people to drive around Cape Town, Bengaluru, Shanghai, Shaanxi, Jakarta, and whichever place that has a lot of "PoC" and kids.
- elif 3y agoThey should have controlled with a person painted blue or purple to determine whether it is simply a matter of photonics rather than AI bias.
- Horffupolde 3y agoIf skin colors differ how can you possibly expect to obtain the same result?
- nick_ 3y agoLIDAR
- bigfryo 3y agoIt's almost as if the media is looking for racism in everything
- ChatGTP 3y ago...racism against little kids?
- endymi0n 3y agoIn other words: "Objects with less surface area and lower albedo are less reliably picked up by visual deep neural nets. We haven't benchmarked this against human drivers."
- tetromino_ 3y agoHuman driver eyes (and I suspect any other optical systems working in the visible color range) are also less likely to detect people of color. Five years ago I avoided running over a pedestrian at night only by luck: he was black, wearing a black jacket, black pants, walking across a badly-lit suburban street; I think that either my visual system did not perceive him at all until the last fraction of a second, or perhaps perceived him as a shadow. I managed to swerve. But a fraction of a second later? I am afraid to think about it... I am a big fan of Scandinavian style pedestrian safety reflectors. Attach one to your bag or jacket if you are walking late at night; it might save your life. But if you don't have a reflector, wear at least one piece of bright, light-colored clothing; this is particularly important it your skin color is dark!
- zephrx1111 3y agoApparently the media wanted to sell an optical problem as racial problem. Some saying: racists are people that are thinking of race, talking about race, and acting based upon race.
- joker_minmax 3y agoRegardless of race, it shouldnt be on the road if it cannot detect jaywalkers at night wearing all black. Where I live POC are more likely to be the pedestrians because it's an immigrant-heavy community. I've seen a white guy jaywalking at night wearing all black in between street lights but I couldn't tell he was white until he got across the street and I could see him head-on.
- sbuttgereit 3y agoThis seems like a pretty poor article on a pretty poor research subject. The way I read it is something like this... Some researchers got their hands on software that purports to do similar stuff to what self driving cars might also do, but crucially isn't the same as what the cars actually use, and then extrapolate the results into the headline-like title of the research paper: "Dark-Skin Individuals Are at More Risk on the Street: Unmasking Fairness Issues of Autonomous Driving Systems". That's justified isn't it? After all, all software in a category is more or less the same program and the car company software and their research subject software all runs on computers? Right? Must be valid... clearly you can make factual assertions on that kind of extrapolation about computer systems and software. Then some bright-eyed-bushy-tailed reporter comes along and applies the criticality of the typical college educated/professional journalist, which is to say they carefully considered the headline they could write, but otherwise just took the word of the researchers that something resembling knowledge was actually gained by the study. News is delivered! Job done! Look, sarcasm aside, could I have read/understood things incorrectly? Sure... I'm not an expert in this field. Could this be a problem in production-used-in-the-real-world pedestrian detection systems? Sure. But insofar as I can tell, the best the paper could be telling us is that racial biases in pedestrian detection systems is a viable possibility: not the assertion that "Dark-Skin Individuals Are at More Risk on the Street". It might be true, but I don't think these researchers know that any better than I do. Of course, "Dark-Skin Individuals Could Be at More Risk on the Street" isn't nearly so catchy or attention grabbing, is it? And who knows... maybe this research team should pick up the search for low temperature/low pressure super-conductivity... sounds like they have the right temperament.
- epivosism 3y agoOne good bit of news is that I can confidently predict that at least they don't follow the sizeism of our society, and are likely able to detect larger people more easily than skinnier.
- archo 3y agohttps://archive.is/8rf7Z https://archive.is/8rf7Z