4 ms·

Things I want to know: - Show us the side-by-side images of the false-positives. Are the matches plausible? - What is the demographic distribution of the mugs

Things I want to know: - Show us the side-by-side images of the false-positives. Are the matches plausible? - What is the demographic distribution of the mugshot database? If the data is disproportionately biased, then that bias would be reflected in the false-positives. A casual skimming of some mughot websites shows a potentially significant racial bias.

hanselot 8y ago
How dare you question our great overlords the ACLU. They would never use statistics to find results favourable to their hypothesis.
- dang 8y ago
  Maybe not, but please don't post unsubstantive comments here.
jerf 8y ago
I have a ton of quibbles with the article and methodology, honestly. I suspect it's just straight-up so flawed as to be irrelevant to the discussion. At the same time, they do still have the correct final conclusion. Dragnet facial recognition is a bad idea that will produce too many false positives to be of use. Or at the very least, dragnet facial recognition used by people who put too much faith in it is a bad idea. It's at most useful to produce flags for slightly more attention; on its own, I would not consider it anywhere close to enough to arrest someone on. And it can serve as a demonstration of how a clumsily-put-together system can be used to produce bad results, so, if we correctly assume that there may be people bodging together a system in much the same way the ACLU did, well, hey, it's a valid result then!
- Kalium 8y ago
  I really want to see their methodology. I'd love to replicate it and play with certainty levels to see what happens.
mrweasel 8y ago
Depending on the technologi behind Amazons facial recognitions, there's very real chance that the software simply associated some black facial features with "being criminal". If that's the case we're back to craniometry in the 1910-1940. Perhaps lawmakers should consider implementing rules to ban facial recognition software until the developers can prove some like 99.999% accuracy. Otherwise we end up harassing a large number of people, and the credibility of the technology will stuff. As it stand any lawyer will be able to argue that facial recognition isn't admisible as evidence, do to the high margin of error.
- Kalium 8y ago
  They fed it a large number of mugshots. They didn't ask "is this person a criminal". They asked it "does this face match one of the faces you've seen before". You're right that systems have previously learned to infer black -> criminal. It's absolutley possible that could have happened here! Given that not every one flagged is a person of color, there's clearly more at work. Systems like Rekognition let a user specify what level of certainty the system has that it has found a match. In this instance, the system was told to use 80%. Rekognition can be told to use any number from 0 to 100. Your number of 99.999% could be used without any difficulty whatsoever! Since the ACLU doesn't seem to be sharing enough to reproduce the test, we can't tell what would happen. Regulation based on accuracy is a wonderful idea! One well worth pursuing. With that said, "accuracy" is perhaps difficult to define here. How would you determine that? How accuracy is measured is highly sensitive to what you use as a dataset, and different test datasets can easily produce very different impressions of how common false positives and false negatives are.
  - StanislavPetrov 8y ago
    They didn't tell it to use 80%, 80% is the default setting. Anyone who thinks that many law enforcement agencies aren't going to use the default setting ought to go look at their routers and other hardware and see how they are configured.
    - Kalium 8y ago
otakucode 8y ago
When facial recognition fails, it usually looks pretty weird. Like it will identify faces in the knots and grain patterns of a wooden wall behind somebody. It's really a very ineffective technology and it's disturbing that it gets sold as something you could use to justify locking someone in a cage over. Machines with pareidolia are not good tools.
- zo1 8y ago
  When people speak of "facial recognition", they speak of faces being compared to other faces. What you're mentioning is face-detection. Are you honestly saying that police departments will have a "criminal database" filled with wooden walls and furniture?
  - otakucode 8y ago
    I'm saying that police departments DO have databases filled with images that have wooden walls and such. Not furniture usually, but photos are often taken in sub-optimal circumstances even when the police or others are trained to take the photos properly. I know this is the case because I've worked on biometric systems for the past 18 years, and I have personally worked on a system which was identifying almost every face to a mugshot taken against a wall with wooden paneling and the matching algorithm was keying in on it as a face rather than the human. I'm not speaking theoretically, this is how these systems fail.
sudhirj 8y ago
Whether the matches are plausible is completely besides the point. This isn't an evaluation of Amazon's recognition quality, it's a demonstration that the system is fallible and notes on the consequences. Same thing re data setup. Not the point again. Even if law enforcement hires the world's best computer vision experts (fat chance, it'll go to cheapest contractor) this is a note that there can be mistakes that get people killed.
- petermcneeley 8y ago
  What are the consequences?? How is this any different than the suspect sketches? This is a far worse situation and still quite minor https://www.cbc.ca/news/canada/british-columbia/bc-mother-child-no-fly-list-1.4390544 https://www.cbc.ca/news/canada/british-columbia/bc-mother-ch...
  - HeyLaughingBoy 8y ago
    One major difference is that non-technical people tend to trust what a computer says far more than they should. People are familiar with sketches and can make judgments on how clearly they match a face. They have no similar frame of reference for computer face recognition.
    - l9k 8y ago
      But people are familiar with photographs. There is no way the software wouldn't show the picture that's been matched before an action (arrest...) is taken
  - sudhirj 8y ago
    What does the link have to do with sketches? It's about name similarity on the no fly list. Regarding sketches, they're actually interesting because they force a human to exercise judgement, being ambiguous by nature. This is fine - any law enforcement officer stopping someone who looks like a sketch is going to give them the benefit of the doubt. On the other hand, if a computer tells an officer a person ahead is an 89% match with a known killer, the conversation will start with guns drawn, and that will greatly increase the likelyhood of things going south. You really don't want to be scanned by a trigger happy cop having a bad day. God knows who you happen to look like from that angle.

hanselot 8y ago

How dare you question our great overlords the ACLU. They would never use statistics to find results favourable to their hypothesis.

dang 8y ago
Maybe not, but please don't post unsubstantive comments here.

jerf 8y ago

I have a ton of quibbles with the article and methodology, honestly. I suspect it's just straight-up so flawed as to be irrelevant to the discussion. At the same time, they do still have the correct final conclusion. Dragnet facial recognition is a bad idea that will produce too many false positives to be of use. Or at the very least, dragnet facial recognition used by people who put too much faith in it is a bad idea. It's at most useful to produce flags for slightly more attention; on its own, I would not consider it anywhere close to enough to arrest someone on. And it can serve as a demonstration of how a clumsily-put-together system can be used to produce bad results, so, if we correctly assume that there may be people bodging together a system in much the same way the ACLU did, well, hey, it's a valid result then!

Kalium 8y ago
I really want to see their methodology. I'd love to replicate it and play with certainty levels to see what happens.

mrweasel 8y ago

Depending on the technologi behind Amazons facial recognitions, there's very real chance that the software simply associated some black facial features with "being criminal". If that's the case we're back to craniometry in the 1910-1940. Perhaps lawmakers should consider implementing rules to ban facial recognition software until the developers can prove some like 99.999% accuracy. Otherwise we end up harassing a large number of people, and the credibility of the technology will stuff. As it stand any lawyer will be able to argue that facial recognition isn't admisible as evidence, do to the high margin of error.

Kalium 8y ago
They fed it a large number of mugshots. They didn't ask "is this person a criminal". They asked it "does this face match one of the faces you've seen before". You're right that systems have previously learned to infer black -> criminal. It's absolutley possible that could have happened here! Given that not every one flagged is a person of color, there's clearly more at work. Systems like Rekognition let a user specify what level of certainty the system has that it has found a match. In this instance, the system was told to use 80%. Rekognition can be told to use any number from 0 to 100. Your number of 99.999% could be used without any difficulty whatsoever! Since the ACLU doesn't seem to be sharing enough to reproduce the test, we can't tell what would happen. Regulation based on accuracy is a wonderful idea! One well worth pursuing. With that said, "accuracy" is perhaps difficult to define here. How would you determine that? How accuracy is measured is highly sensitive to what you use as a dataset, and different test datasets can easily produce very different impressions of how common false positives and false negatives are.
- StanislavPetrov 8y ago
  They didn't tell it to use 80%, 80% is the default setting. Anyone who thinks that many law enforcement agencies aren't going to use the default setting ought to go look at their routers and other hardware and see how they are configured.
  - Kalium 8y ago

otakucode 8y ago

When facial recognition fails, it usually looks pretty weird. Like it will identify faces in the knots and grain patterns of a wooden wall behind somebody. It's really a very ineffective technology and it's disturbing that it gets sold as something you could use to justify locking someone in a cage over. Machines with pareidolia are not good tools.

zo1 8y ago
When people speak of "facial recognition", they speak of faces being compared to other faces. What you're mentioning is face-detection. Are you honestly saying that police departments will have a "criminal database" filled with wooden walls and furniture?
- otakucode 8y ago
  I'm saying that police departments DO have databases filled with images that have wooden walls and such. Not furniture usually, but photos are often taken in sub-optimal circumstances even when the police or others are trained to take the photos properly. I know this is the case because I've worked on biometric systems for the past 18 years, and I have personally worked on a system which was identifying almost every face to a mugshot taken against a wall with wooden paneling and the matching algorithm was keying in on it as a face rather than the human. I'm not speaking theoretically, this is how these systems fail.

sudhirj 8y ago

Whether the matches are plausible is completely besides the point. This isn't an evaluation of Amazon's recognition quality, it's a demonstration that the system is fallible and notes on the consequences. Same thing re data setup. Not the point again. Even if law enforcement hires the world's best computer vision experts (fat chance, it'll go to cheapest contractor) this is a note that there can be mistakes that get people killed.

petermcneeley 8y ago
What are the consequences?? How is this any different than the suspect sketches? This is a far worse situation and still quite minor https://www.cbc.ca/news/canada/british-columbia/bc-mother-child-no-fly-list-1.4390544 https://www.cbc.ca/news/canada/british-columbia/bc-mother-ch...
- HeyLaughingBoy 8y ago
  One major difference is that non-technical people tend to trust what a computer says far more than they should. People are familiar with sketches and can make judgments on how clearly they match a face. They have no similar frame of reference for computer face recognition.
  - l9k 8y ago
    But people are familiar with photographs. There is no way the software wouldn't show the picture that's been matched before an action (arrest...) is taken
- sudhirj 8y ago
  What does the link have to do with sketches? It's about name similarity on the no fly list. Regarding sketches, they're actually interesting because they force a human to exercise judgement, being ambiguous by nature. This is fine - any law enforcement officer stopping someone who looks like a sketch is going to give them the benefit of the doubt. On the other hand, if a computer tells an officer a person ahead is an 89% match with a known killer, the conversation will start with guns drawn, and that will greatly increase the likelyhood of things going south. You really don't want to be scanned by a trigger happy cop having a bad day. God knows who you happen to look like from that angle.