4 ms·
> "We believe this results from factors that include the lack of Black faces in the algorithms' training data sets..." the researchers wrote in an op-ed for Sci
by cltby 3y ago
> "We believe this results from factors that include the lack of Black faces in the algorithms' training data sets..." the researchers wrote in an op-ed for Scientific American.
> The research also demonstrated that Black people are overrepresented in databases of mugshots.
The sort of clear-headed thinking that makes the AI bias field as respected as it is.
- sonofhans 3y agoYeah, that’s terrifying. It’s the same reasoning as, “Of course more Black people are convicted of crimes — we arrest more of them!”
- wuiheerfoj 3y agoBut presumably the AI is just identifying faces, not judging who is and is not a criminal
- vkou 3y agoIf the training set is biased, the AI becomes biased. GIGO.
- Tozen 3y agoWhen this software is being sold to these departments, it's amazing that people in the chain don't seem to be talking enough about the training set used or performance on certain populations. If you are going to arrest or build a case on facial recognition, you would think that they would be prepared to defend its accuracy against a broad range of demographics. Embarrassing failures and mistaken arrests, hurts their program, not to mention the money the city losses in lawsuits.
- vkou 3y agoThe answer to this conundrum might be that neither the departments nor the vendors are particularly interested in avoiding bias. Paying lip service is generally sufficient.
- memefrog 3y ago[flagged]
- stormfather 3y agoIf we can't bring up the elephant in the room how are we ever going to solve this issue?
- marsven_422 3y ago[flagged]
- GloomyBoots 3y agoCome on, we know that there is variation, sometimes drastic, between populations on all different facets of life. But this one? No. It would be racist to even broach the subject. That’s why we know White people are to blame for it.
- lozenge 3y agoIt makes sense to me? The algorithm specialises in distinguishing between the faces in its training set. It works by dimensionality reduction. If there aren't many black faces there it can just dedicate a few of its dimensions to "distinguishing black face features". Then if you give it a task that only contains black faces, most of the dimensions will go unused.
- cltby 3y agoAre black faces overrepresented or underrepresented? According to AI researchers, we're faced with Schrodinger's Mugshot--there's simultaneously too many and too few!
- BoorishBears 3y agoThere's two datasets but your conviction that there's some political undertone to this is leaving you unable to process basic logic.
- cltby 3y agoWith a moment's thought, even the most emotive amongst us should see that the mugshots will be part of the training set--the photographed individuals are, after all, the class of true positives.
- codetrotter 3y agoYou train a model on a bunch of photos of white people, and a few photos of black people. You then deploy that model, and use the model to match black person detained by racist officers against a database of photos that the police have from before. In that database the majority of people are black. Shitty AI that was not properly taught what black people look like because most of the people in the training data were white, says that it found a probable match for detained black person. Racist officers do not attempt to second guess the computer, so they throw innocent black person into their car and drive off to the police station.
- 3y ago
- zerocrates 3y agoThe actual quote that the mention in the article refers to: "Using diverse training sets can help reduce bias in FRT performance. Algorithms learn to compare images by training with a set of photos. Disproportionate representation of white males in training images produces skewed algorithms because Black people are overrepresented in mugshot databases and other image repositories commonly used by law enforcement. Consequently AI is more likely to mark Black faces as criminal, leading to the targeting and arresting of innocent Black people." So they're saying that simultaneously the training set has too few black faces and the set being compared against has too many.
- cltby 3y agoThis doesn't rescue their claim. If the suggested class imbalance really exists in the training/test sets, the model will preferentially identify whites as criminals.
- zerocrates 3y agoThe claim is that the model is worse at telling black faces apart from each other. The system is trained to match images of faces, not identify criminals; it's not comparing things to its training set to give a "criminality" score. The training data is just what has taught the system how to extract features to compare. You run an image of an unknown person against your database of known images, and look for a match so you can identify the unknown person. If the model is just "worse at" black people, it's going to make more mistakes matching to them.
- GloomyBoots 3y ago> Consequently AI is more likely to mark Black faces as criminal, leading to the targeting and arresting of innocent Black people. I don’t see how this relates to simple facial recognition. It doesn’t appear that they’re scanning for “criminal physiognomies” but for specific facial matches. Furthermore, it seems that this whole line of argumentation implies that facial recognition software may be mistaking innocent Black people for non-Black perpetrators, which I don’t see any evidence for. How does this increase arrest rates for Black people if AI just can’t tell them apart? In all likelihood, the person who got away is also Black.