13 ms·
Federal study confirms racial bias of many facial-recognition systems
- formercoder 7y agoDarker skin has less contrast between highlights and shadows so isn’t it a more difficult problem to identify them from a technical perspective?
- deleted 7y ago[deleted]
- cameronfraser 7y agoWell the consensus from the article is that middle aged white men had the best rate of success with facial recognition which sounds like a data mismatch problem to me.
- shadowgovt 7y agoIt seriously depends on the technology being used. There isn't any evidence from human psychology that dark-skinned people have harder-to-recognize faces, so we're probably just using the wrong approach in the technology currently available.
- formercoder 7y agoGreat point, I don’t know how much our brain uses, for example, depth information in facial recognition.
- jlawson 7y agoEyes are just better at distinguishing similar dark colors against each other compared to cameras. Only the most expensive, massive and heavy cameras (costing hundreds of thousands of dollars, generally) can compete with eyes in this respect. Plus human facial recognition is massively overpowered for normal cases. I think if you tested it in challenge cases (through blurry video, or snow, or in the dark, or at a distance), you might see darker skin tones being recognized less easily. It'd be an interesting experiment.
- shadowgovt 7y agoThat might very well be true, but if it is, then it re-enforces the point: the technology may not be where it needs to be yet, and if it isn't, the tool probably shouldn't be deployed in public use cases where the end-result could be inherently discriminatory by race.
- aaron695 7y ago>There isn't any evidence from human psychology that dark-skinned people have harder-to-recognize faces Citation? Dark cars have more accidents- https://www.telegraph.co.uk/motoring/news/7845366/Black-cars-more-likely-to-be-involved-in-crashes.html https://www.telegraph.co.uk/motoring/news/7845366/Black-cars... There clearly will be a difference. Whole careers are around making things more or less recognizable depending on color. What study makes you think it's insignificant for human faces?
- shadowgovt 7y agoWhat study makes you think it is significant for human faces? I'm noting absence of evidence; if you're making the point that it doesn't imply evidence of absence, I agree, but I've also never heard anyone seriously claim "black people's faces are just harder to recognize." The anecdotal claim isn't even present. The linked study on cars is analogous at best, but doesn't prove anything for human facial recognition. Cars on highways and human beings in various settings are extremely different circumstances.
- SiVal 7y agoPart of the technology is the capturing of photons, fewer of which bounce off dark skin than light skin, so it is in part the same generic problem as always: fewer photons, less data, worse images. Whether it's day vs. night, fast lens vs. slow, close vs. distant, fast moving vs slow (requiring fast shutter than lets in less light), or whatever causes it, fewer photons mean more problems. And even if you get enough photons to, for example, recognize both black and white equally well in daylight, you might not at night or in a deep shadow indoors, or up close equally well but there will be some distance at which the advantage of more photons begins to matter, or some speed, or some combination of factors. Any technological improvements will help, but there won't be any technological solution that will always work equally well with fewer photons as with more.
- ummonk 7y agoCameras don't have the same dynamic range as human eyes.
- ailideex 7y agoMaybe there is some transitive effect from racially biased cameras.
- notauser 7y agoThe very best HDR cameras have roughly the same dynamic ranges as the human eye. You can't display all of this range to people because of the limitations of display technology, but you can feed the full range to a facial recognition engine. Security budgets might not stretch to top-end HDR equipment but the price keeps on coming down. The performance of a modern flagship phone is remarkable compared to a few years ago - and fixed surveillance cameras can have much bigger glass and sensors, making it cheaper to get super-human performance. One new but related issue is that body-worn cameras can capture more low-light detail than the human eye. Police unions have argued against deploying these sensors, because, they want the evidential record to show what the officer could see - not what a cat could see.
- randomcarbloke 7y agoMost digital cameras have a dynamic range well in excess of the human eye.
- etaioinshrdlu 7y agoLess reflected light, all else being equal, means the same amount of detected noise (whether using cameras or eyes), and therefore a worse signal to noise ratio on the dark image. Worse signal to noise ratio means less information. Less information indicates worse detection accuracy. While there isn't evidence that dark-skinned people have harder-to-recognize faces, there is also no evidence to the contrary. It seems like the null hypothesis in this case, given no additional evidence, is to assume that darker images are indeed harder to recognize.
- kevingadd 7y agoTo a degree this comes down to how you capture an image of their skin. Lighting has a big impact on how the result looks as does the medium you use to capture the image. In the early days of color photography there was an issue with some films and reference images being tuned for the most common subject (light-skinned humans) and as a result if you tried to capture a mix of races you'd get bad results: https://petapixel.com/2015/09/19/heres-a-look-at-how-color-film-was-originally-biased-toward-white-people/ https://petapixel.com/2015/09/19/heres-a-look-at-how-color-f... This makes sense if you consider how light is (generalizing here) a broad spectrum of hues and a given material is most reflective for specific parts of the spectrum, so if you don't capture much there you'll get a low-contrast image, like stripping the R channel out of an RGB bitmap. It's of course possible to solve the problem for a wider set of skin tones, and it has been solved, but it takes more work. It's a subject of ongoing discussion/experimentation in film to this day: https://www.konbini.com/en/cinema/insecure-cinematographer-how-light-black-skin/ https://www.konbini.com/en/cinema/insecure-cinematographer-h...
- rolltiide 7y agothe point is that these products would not have shipped at all if their workforces were diverse enough to point out “hey this doesn't work, like at all” or “hey this glaring edge case isn't far enough on the edge to send this to production”
- okusername 7y agoThat's a bold assumption. Such algorithms are developed wwith training sets, not just by employees randomly clicking around. They wrote that 189 algorithms were submitted, there were surely a few developed by non-racist, diverse people.
- rolltiide 7y agothe hyperbole is not really helpful or substantive, its not about the algorithm its about how the product wouldnt have shipped when individual contributors, their manager, the director, the executive team, the board, and their vendors all noticed that it didnt work very well when the demo failed because they were not white males testing it on themselves and their colleagues and photos of their friends
- omar_a1 7y agoIt's the methods being used to select, populate, label, and validate the training set that are the problem. Basically, your team is composed of white dudes who don't see the problem with a ML training set consisting largely of pictures of white dudes. To prevent this they'd have needed to A) Employ a black person, and B) Listen to said employee's feedback, in order to recognize the problem. Edit: Also worth pointing out, just using a representative population sampling would still show racial bias, essentially weighting accuracy with respect to population percent. You'd probably need to have equal samplings of pictures of people from all races/genders/disabilities if you wanted equal accuracy across the board. That also includes picture quality and range of picture quality. Doubling up images, or using corporate headshot white dudes and grainy selfie People of Color could still cause issues. Same logic applies to labelling. That minimum wage contracting firm used to decide who's who in the photos may exhibit racial bias, by virtue of the fact that most people do. If their accuracy in labelling is racially biased then so too will the algorithms that it's based on. In short: Racist garbage in, racist garbage out.
- deleted 7y ago[deleted]
- trianglem 7y agoDoesn’t this depend on the backdrop.
- formercoder 7y agoComplicated answer. Theoretically no, I was just mentioning highlights vs shadows on the face which are properties of reflectivity. The backdrop certainly matters from a camera exposure perspective in order to optimize for the skin properties in play.
- throwawaysea 7y agoThe article casts doubt on expanded use of facial recognition but that’s not the right reaction IMO. Facial recognition systems are used as a first pass filter. That is they are reducing a large space of possible matches to a lesser space of possible matches. It is significantly better than expecting beat cops to match faces on the sidewalk as they look around, for example. No one, to my knowledge, is relying on facial recognition systems as absolute matches. The ACLU rep that is quoted is IMO relying on hyperbole to make his point (“One false match can lead to missed flights, lengthy interrogations, tense police encounters, false arrests, or worse”). If the match is close enough that a human would confirm it as well, and conduct an “interrogation”, then presumably facial recognition technology did not incrementally add to the problem of mismatching since a human would also make the same false match. So if it makes policing more efficient and enables more criminals to be nabbed, and it is used as a first pass filter (so the false positive rate doesn’t matter), I am all for it.
- nyolfen 7y ago> If the match is close enough that a human would confirm it as well, and conduct an “interrogation”, then presumably facial recognition technology did not incrementally add to the problem of mismatching since a human would also make the same false match. it seems likely to me that cops might use it as justification for a fishing expedition, or TSA agents may just go along with it as a cover-your-ass measure
- SlowRobotAhead 7y ago“Sir do I smell marijuana in your vehicle?” “Sir, this dog detected something on your person” “Sir, the scanned detected an object under your shirt please step over here” “Sir, you’ve been flagged by this facial recognition software” ... agreed. I think this is better to head off before that last one can be abused.
- golemiprague 7y agoWhy they have to use "racial bias" immediately? Can't they just treat it like a normal stage of a developing of a system? All systems are developed with some "bias" until they are used a lot and all the "biases" are smoothed out. The article says itself that in Asian countries they are better at recognising Asian people, so it is only normal progression, you start with the people around you and then slowly improve the system, I don't see what is the problem with it. How about black people just develop a better system which will deal better with black face features instead of whinging and crying about their poor luck? The same thing the Asians are doing, I am sure law enforcement agencies will be happy to buy a better system if exists.
- daenz 7y agoHere's the study, which curiously isn't linked to by WP https://nvlpubs.nist.gov/nistpubs/ir/2019/NIST.IR.8280.pdf https://nvlpubs.nist.gov/nistpubs/ir/2019/NIST.IR.8280.pdf
- daenz 7y agoThe article seems to hang on the false positives being higher for certain minorities, but makes no mention of the false negatives being higher as well. So while some ethnic minorities might be more likely to match incorrectly to a known person of interest, they're also more likely to be let through if they are indeed that person. I think that the first case is definitely more damaging on the whole, but I still find it misleading to not mention the specific scope of the "racial bias."
- Barrin92 7y agoit doesn't seem relevant. In a country like the US the law pursues criminal individuals, not criminal groups. The fact that the system also falsely negatively identified people doesn't somehow 'balance out' false positives, it's not like the point of these systems is to randomly catch 100 African-Americans. (although given the political ideologies of some of the people in the surveillance tech space maybe it is) The fact that it produces errors in both directions just makes the system even worse in total.
- shadowgovt 7y agoExactly. If anything, it means the problem is worse than the article indicates.
- daenz 7y ago>The fact that it produces errors in both directions just makes the system even worse in total. It's almost like the purpose of the article isn't to show the tech is overall empirically bad.
- allovernow 7y ago>The fact that the system also falsely negatively identified people doesn't somehow 'balance out' false positives, it's not like the point of these systems is to randomly catch 100 African-Americans. If they're disproportionatly committing crimes, skin color becomes a strong prior. It would be rational, iff the goal is to lower crime rates. But I don't know that it is morally OK. Also I'm not sure if over representation is the source of bias in this particular case, haven't read the article yet. Edit: yeah, this article is just a smear, wapo is salivating over anything they think confirms that police are racially biased. It doesn't mention anything technical about the training data or typical nets being used. I'm not advocating for facial recognition tech, I'm personally against it, but racial bias coming from objective data is the least of your potential worries.
- surround 7y agoThe technology will inevitably improve. Real issues will arise when facial-recognition becomes perfectly accurate (without false positives, racial bias, etc). Governments will be able to track citizens anywhere.
- swiley 7y agoWhat reason do you have to think it will improve to be perfectly accurate?
- surround 7y agoThe point I was trying to make is that any facial recognition - racial biased or not - is an issue.
- joshuamorton 7y agoAs written, you seem to be implying that the current issues aren't real. They certainly are and affect people now.
- surround 7y agoCorrect. I should’ve said “larger” issues.
- crooked-v 7y agoOn the other hand, some governments may find it extremely convenient to use 'accidentally' biased technology to harass and persecute members of disliked minorities.
- ydb 7y agoDoesn't this also prove, by proxy, that this fact can potentially (if not certainly) apply to other AI & associated fields? Forgive my ignorance, but this seems to lend credence to the popular idpol claim that the white guys programming AI are ignorant of the inherent bias their models might have.
- zadokshi 7y agoYou are assuming that AI's are only programmed by white men, and you are assuming that it hasn't occurred to anyone who is an expert in the field of facial recognition that their algorithm's might be used with people other than white people. These are two bold assumptions. Would you really have us believe that this is the case? I think you will find that some important information has been left off. The article suggests that asian created algorithms are better at recognising asians. It implies that they all have problems with darker skin tones. Why does the article not explore the reasons why? Perhaps they are wanting to say that algorithms can be racist, rather than the truth of the matter, which is there are certain technical issues that are difficult to overcome with darker skin tones.
- 6700417 7y ago> than the truth of the matter, which is there are certain technical issues that are difficult to overcome with darker skin tones. Can you point to any references supporting this claim?
- dahfizz 7y agoAn article about the technicallities of cameras and facial recognition on darker skin tones would generate a lot less clicks than "Computers are racist!"
- smt88 7y agoI think that "less accuracy for a certain race" + "facial recognition used for finding criminals" is where the word bias (notably "bias" and not "racism") comes from. The first sentence of the article backs up its usage of the word bias instead of inaccuracy: > "...casting new doubts on a rapidly expanding investigative technique widely used by law enforcement across the United States." emphasis mine
- deleted 7y ago[deleted]
- wojcikstefan 7y agoSurprise surprise, in a field dominated by white men, the technology is deemed production-ready when it’s accurate for white men. Not blaming this group or anyone in particular – we all just need the be cognizant of the fact that we have our blind spots (racial, gender, and otherwise).
- commandlinefan 7y ago> a field dominated by white men Actually, as far as I can tell, it's dominated by asians - specifically Indians.
- wojcikstefan 7y agoOf course, accurate data is hard to come by, but this report from Reveal from The Center for Investigative Reporting suggests otherwise, especially at executive levels: https://www.revealnews.org/article/heres-the-clearest-picture-of-silicon-valleys-diversity-yet/ https://www.revealnews.org/article/heres-the-clearest-pictur...
- shadowprofile77 7y agoThe points that this article and many of the comments here seem to make about the dangers of these racially-caused mistakes miss the more fundamental problem of this technology being used ever more widely in the first place. Facial recognition will invariably get better (virtually certainly to the point of almost completely avoiding mistakes based on a persons ethnic makeup) and that's what's really terrible, because it hints at the sort of constant surveillance future we have coming toward us. If anything, it's almost comforting to know that right now, at the very least, these algorithms don't yet work well enough to avoid politically unfortunate errors, because the political consequences of these errors slow down the scenario of total, nearly error-free ubiquity that anyone who even slightly values privacy should be very worried about.
- raxxorrax 7y agoThe inherent problem facial recognition and similar systems have is that they just rely on an extremely narrow context. But I agree that it is not constructive to points towards bias (in many discussions actually), since that will just normalize the presence of the technology itself. Suddenly it just needs to solve the problem of improving some statistics instead of justifying the deployment of cameras everywhere in the first place. So the criticism is a particularly bad one that omits to ask the hard questions.
- tu7001 7y agoCorrect, spread the word!
- menzoic 7y agoI would agree if these systems weren't being used to accost and prosecute people in the affected groups in the real world today. Considering that humans are willing to blindly follow these false results I'd say it's even more frightening rather than comforting.
- ojagodzinski 7y agopaywall. $30 for two paragraphs?
- CryptoPunk 7y agoAny heuristic will have varying effects with respect to different groups of individuals, regardless of what property individuals are being grouped by. It's absolutely irrelevant whether the heuristic is unfair to individuals of a particular race, or unfair to individuals of a particular nose length. Prioritizing the former over the latter is a totally arbitrary value system that deprioritizes the most important metric for maximizing fairness, which is the overall accuracy rate. Ultimately any inaccuracy of the heuristic is an instance of the heuristic treating someone unfairly. The objective should be to minimize the inaccuracy rate overall, not the inaccuracy rate in relation to politically prioritized groupings like race.
- sputknick 7y agoit looks to me like they buried the lede in the last paragraph. 'relationship “between an algorithm’s performance and the data used to train it,”' To solve this problem they should look at the relationship between the number of pictures for a given race, and the accuracy in recognizing members of that race. It probably does best at identifying European descendant faces because it was developed in a country where European descendants are the largest ethnic group. I'd bet dollars to donuts that if you can increase the number of faces for each race this disparity will largely disappear.
- flurbix 7y ago"Algorithms developed in Asian countries had smaller differences in error rates between white and Asian faces, suggesting a relationship “between an algorithm’s performance and the data used to train it,” Quick, somebody offer the guy who realized this a job.
- adriantam 7y agoIn China, you will heard of the company SenseTime who specialize in facial-recognition. I bet if you do the same study on that system, you will see the same racial bias on white people! All the problem is data. Try train on man faces and apply it on women faces. If you want to solve this problem, gather more training data on those "minorities" -- even if that means you need to go to a foreign country to collect them.
- jokesnotfunny 7y agoInequality comes from nature tho, what has racial bias to do with anything? It just used your facial features right. Should blame our genes and ancestors for bias, not technology.