3 ms·

Facial recognition has a disturbing habit of "catching" the wrong people. "We have video of a crime, arrest these people", is how facial recognition is adverti

Facial recognition has a disturbing habit of "catching" the wrong people. "We have video of a crime, arrest these people", is how facial recognition is advertised (and you bought the story). But it keeps being used as "there is a <19 times in 20> 87% chance that this person walking down the street in Atlanta is the person who robbed a liquor store in Moose Jaw, Saskatchewan in 2006, go <violently> arrest that guy"

Jkvngt 6y ago
Is this actually true though? It sounds great, but where’s the data? * we need more than just the occasional anecdote to determine if this is a real issue
- keiferski 6y ago
  https://www.nytimes.com/2020/06/24/technology/facial-recognition-arrest.html https://www.nytimes.com/2020/06/24/technology/facial-recogni...
- Ensorceled 6y ago
  There are literally dozens of news stories about the wrong person being arrested for a crime they could not have committed. It's literally why cities have been banning the practice in the original news story.
- giardini 6y ago
  Facial recognition software fails legal tests if used with large datasets. It does so b/c software does not discriminate in the same way the human mind does. For example, some facial recognition software identifies this picture of a man wearing glasses as actress Milla Jovovitch: https://images.newscientist.com/wp-content/uploads/2016/11/01160657/unknown.jpg?width=800 https://images.newscientist.com/wp-content/uploads/2016/11/0... [The New Scientist article from which the above picture is taken is "Glasses make face recognition tech think you're Milla Jovovich": https://www.newscientist.com/article/2111041-glasses-make-face-recognition-tech-think-youre-milla-jovovich/ https://www.newscientist.com/article/2111041-glasses-make-fa... ] The (PDF) original paper: https://www.cs.cmu.edu/~sbhagava/papers/face-rec-ccs16.pdf https://www.cs.cmu.edu/~sbhagava/papers/face-rec-ccs16.pdf Steven Talley was identified by the FBI as the primary suspect in two bank robberies using a facial recognition algorithm. He had an iron-clad alibi, but the police and FBI weren't convinced. In court one of the bank tellers said Talley definitely wasn't the robber. Nonetheless Talley lost his job, his wife and his family and was held in prison for months. For more details read: "LOSING FACE: How a Facial Recognition Mismatch Can Ruin Your Life" https://theintercept.com/2016/10/13/how-a-facial-recognition-mismatch-can-ruin-your-life/ https://theintercept.com/2016/10/13/how-a-facial-recognition... I found out about the facial recognition failures from he outstanding book "Hello World" by Hannah Fry. Fry tells the story of Steven Talley as part of a chapter on crime, AI and facial recognition. https://www.amazon.com/Hello-World-Being-Human-Algorithms/dp/039363499X https://www.amazon.com/Hello-World-Being-Human-Algorithms/dp... Fry's book shows how/why facial recognition software simply does not work well enough to use in police work. She provides the studies and footnotes them. As Fry says: "If you're searching for a particular criminal in digital line-up of millions...the best-case scenario is that you won't find the right person one in six times...". That is not nearly good enough for law enforcement and the courts. -from "Hello World" by Hannah Fry
- glenda 6y ago
  The police chief in Detroit said their facial recognition system had a 4% success rate. Presumably this means that 96% of the time they were targeting someone with absolutely no relation to the actual crime. What a complete waste of time and disregard for our civil liberties. https://arstechnica.com/tech-policy/2020/06/detroit-police-chief-admits-facial-recognition-is-wrong-96-of-the-time/ https://arstechnica.com/tech-policy/2020/06/detroit-police-c...
bsenftner 6y ago
Facial recognition needs to be paired with a screening for Racial Blindness with any operator of FR software. FR software provides a filtering, given many faces, here are the best matches. This is where the FR software operator is critical because it is then their task to pick the best match. The final step in FR is always a human making the selection, primarily due to a lack of image quality the "best match" cannot be assured to be the correct match, and even then the correct person may not even be in the FR database. This is the type of understanding and analysis that needs to be present in an FR software operator, but as far as I can tell this area is completely ignored by both the producers of FR and the customers of FR. Disclosure: I am the lead developer of a global leading FR solution.
mytailorisrich 6y ago
That's a problem with police procedures and training of those who use facial recognition, not facial recognition per se.
- Ensorceled 6y ago
  Yes? Cities banned facial recognition use because police have proven unable to create effective procedures and training to prevent the problems.
- dhimes 6y ago
  Indeed. Let's not forget, eyewitnesses are notoriously unreliable as well.

Jkvngt 6y ago

Is this actually true though? It sounds great, but where’s the data? * we need more than just the occasional anecdote to determine if this is a real issue

keiferski 6y ago
https://www.nytimes.com/2020/06/24/technology/facial-recognition-arrest.html https://www.nytimes.com/2020/06/24/technology/facial-recogni...
Ensorceled 6y ago
There are literally dozens of news stories about the wrong person being arrested for a crime they could not have committed. It's literally why cities have been banning the practice in the original news story.
giardini 6y ago
Facial recognition software fails legal tests if used with large datasets. It does so b/c software does not discriminate in the same way the human mind does. For example, some facial recognition software identifies this picture of a man wearing glasses as actress Milla Jovovitch: https://images.newscientist.com/wp-content/uploads/2016/11/01160657/unknown.jpg?width=800 https://images.newscientist.com/wp-content/uploads/2016/11/0... [The New Scientist article from which the above picture is taken is "Glasses make face recognition tech think you're Milla Jovovich": https://www.newscientist.com/article/2111041-glasses-make-face-recognition-tech-think-youre-milla-jovovich/ https://www.newscientist.com/article/2111041-glasses-make-fa... ] The (PDF) original paper: https://www.cs.cmu.edu/~sbhagava/papers/face-rec-ccs16.pdf https://www.cs.cmu.edu/~sbhagava/papers/face-rec-ccs16.pdf Steven Talley was identified by the FBI as the primary suspect in two bank robberies using a facial recognition algorithm. He had an iron-clad alibi, but the police and FBI weren't convinced. In court one of the bank tellers said Talley definitely wasn't the robber. Nonetheless Talley lost his job, his wife and his family and was held in prison for months. For more details read: "LOSING FACE: How a Facial Recognition Mismatch Can Ruin Your Life" https://theintercept.com/2016/10/13/how-a-facial-recognition-mismatch-can-ruin-your-life/ https://theintercept.com/2016/10/13/how-a-facial-recognition... I found out about the facial recognition failures from he outstanding book "Hello World" by Hannah Fry. Fry tells the story of Steven Talley as part of a chapter on crime, AI and facial recognition. https://www.amazon.com/Hello-World-Being-Human-Algorithms/dp/039363499X https://www.amazon.com/Hello-World-Being-Human-Algorithms/dp... Fry's book shows how/why facial recognition software simply does not work well enough to use in police work. She provides the studies and footnotes them. As Fry says: "If you're searching for a particular criminal in digital line-up of millions...the best-case scenario is that you won't find the right person one in six times...". That is not nearly good enough for law enforcement and the courts. -from "Hello World" by Hannah Fry
glenda 6y ago
The police chief in Detroit said their facial recognition system had a 4% success rate. Presumably this means that 96% of the time they were targeting someone with absolutely no relation to the actual crime. What a complete waste of time and disregard for our civil liberties. https://arstechnica.com/tech-policy/2020/06/detroit-police-chief-admits-facial-recognition-is-wrong-96-of-the-time/ https://arstechnica.com/tech-policy/2020/06/detroit-police-c...

bsenftner 6y ago

Facial recognition needs to be paired with a screening for Racial Blindness with any operator of FR software. FR software provides a filtering, given many faces, here are the best matches. This is where the FR software operator is critical because it is then their task to pick the best match. The final step in FR is always a human making the selection, primarily due to a lack of image quality the "best match" cannot be assured to be the correct match, and even then the correct person may not even be in the FR database. This is the type of understanding and analysis that needs to be present in an FR software operator, but as far as I can tell this area is completely ignored by both the producers of FR and the customers of FR. Disclosure: I am the lead developer of a global leading FR solution.

mytailorisrich 6y ago

That's a problem with police procedures and training of those who use facial recognition, not facial recognition per se.

Ensorceled 6y ago
Yes? Cities banned facial recognition use because police have proven unable to create effective procedures and training to prevent the problems.
dhimes 6y ago
Indeed. Let's not forget, eyewitnesses are notoriously unreliable as well.