4 ms·

Did anyone read the story? The victim physically picked the woman out of a lineup of mug shot photos placed in front of him. The software placed the woman’s ph

by CPLX 3y ago

Did anyone read the story? The victim physically picked the woman out of a lineup of mug shot photos placed in front of him. The software placed the woman’s photo in the lineup but the victim of the crime literally ID’d her. Wrongly, of course, but seem like a pretty crucial detail. This is more of an old fashioned police are idiots story than anything else. Like how about investigating instead of just arresting her. Like was she near the crime, did she have an alibi? Etc.

jtbayly 3y ago
I read it. The article gives a perfect description of how it happened. The software picked her because she looked like the criminal, and then they put her in the lineup. They wouldn’t have had a photo lineup without the software. That’s the whole point. They use the fig leaf of the victim ID, to cover the fact that it’s just facial recognition at the core. The article points out how likely somebody is to mis-ID somebody who the computer has already found is a likely match. Sure, blame the police. But for what? For using facial recognition to ID and arrest people!
- MostlyStable 3y ago
  This is actually a really good point. My understanding of traditional line ups (do not take my word for this, at all) is that cops put someone(s) in the lineup because they have reason to believe that person is a suspect for some reason, and then then the victim is supposed to pick them out from among other random people. If the entire lineup is selected to be people who "look like the suspect", and, in fact, that's why every one of them is there, then the victim picking them seems to be A) much sketchier and B) much less valuable. The proper way to use this tool seems to be 1. Find all the people who look similar to the low quality pic we have of the suspect. 2. Investigate those people to see if they could have done it. 3. Arrest someone who you hopefully now have additional evidence against besides just "they look like a grainy photo of the suspect" Just throwing a bunch of people who look like the suspect at the victim and just arresting whoever the victim picks seems completely insane. But also, like it doesn't have too much to do with "Facial recognition" broadly speaking and more to do with completely idiotic investigative practices.
  - jtbayly 3y ago
    I assumed only one of the photos looked like the perp, which changes it quite a bit. Makes it even worse. How is the person supposed to not pick the facial recognized one at that point? At least if all 6 look like the perp, he might have doubt that he can properly ID them.
dundarious 3y ago
Half of the problem with these tools is that software companies have oversold their competence to a wild degree, to the point where it is trusted as much as a DNA test. "The AI said it's her". No further thinking is required, such as thinking about what photo the AI used, whether an 8-months pregnant woman really had sex with this guy before the stick up, etc. Trust the fancy computer. But I also don't think it's too cynical to believe that at this point, almost everybody up and down the chain knows that these tools don't work very well, but what they like is that they are allowed to defer to these tools. It's like how nobody is fired for buying IBM -- nobody is at fault for relying on the algorithm. This applies to many domains, even consumer tech stuff, hiring practices, etc. Algorithms are just a way to have a color by numbers process where employees have no agency and deserve very little pay, and management has no responsibility and cannot be blamed for mistakes.
- woodruffw 3y ago
  DNA testing is also significantly less reliable than the police would have you and I believe[1]. HN applies ample and eager skepticism to all kinds of scientific and science-adjacent fields; I eagerly await the day that we do the same for forensic "science." [1]: https://www.theatlantic.com/magazine/archive/2016/06/a-reasonable-doubt/480747/ https://www.theatlantic.com/magazine/archive/2016/06/a-reaso...
  - dundarious 3y ago
    I haven't read your link yet, but fair point, I am aware there are definitely some issues with DNA testing as practiced by the justice system, but my prior was that when comparing facial recognition to DNA testing, using ideal conditions for both, DNA testing is on far surer footing than facial recognition -- and the ideal conditions for DNA testing are far more attainable than those for facial recognition. For example, I believe the people acquitted based on DNA testing are usually acquitted for good reason, and not just because it makes me feel good, but I'll read your article.
woodruffw 3y ago
I read the story. The fact that the software produced a lineup rather than singling her out is more or less immaterial; what matters is that it was trusted at all (when, as TFA mentions, it appears to be particularly unreliable with pictures of black people). More than one party can share responsibility in a situation, and in this situation both the police and the software should have blame identified with them.
- mapt 3y ago
  The software being trusted to pick out similar faces is a GREAT ADVANCE IN SOLVING CRIMES because it scales the process going on in a photo lineup. But it's only being trusted to pick out similar faces. There will ALWAYS be similar faces. It's doing the same part of the task as a photo lineup does. The point is that there has to be MORE than "This person has a similar face". It's a confirmation step for suspects that already exist, or for whom you can find other strong circumstantial evidence for. You have an unlimited supply of similar faces and you don't get to just choose one of them. The great problems with AI tools mostly arise from the fact that the supposed expert users were never taught what the tools do, or how causal inference works - how to discern reality from the evidence available, how to test hypotheses, how to change your reasoning to be valid in the face of wildly disparate scales. If you don't have at least the beginning of Bayesian probability, and you're feeding large datasets into policing, you have no fucking clue what you're doing and your ignorance is immediately dangerous.
  - woodruffw 3y ago
    No crimes were solved here. We can't take a thing that literally did nothing except cost taxpayer money, police time, and stress for a pregnant woman and declare victory for it. > It's doing the same part of the task as a photo lineup does. It's doing something subtly different, and arguably pernicious: a lineup is meant to contain a variety of individuals of varying similarities, so that the person selecting from the lineup are themselves tested in their recollection. We have no evidence that this system is being used in this way; instead, it seems to be finding the closest match in an existing dataset under the assumption that anybody who's ever been photographed by the police is permanently a criminal suspect.
arp242 3y ago
I just can't get over how being 8 months pregnant (7 and half months at the time of the crime) was not considered an alibi, or at least considered. "Wait, the person who reported the crime didn't mention that, surely he would have noticed while they were drinking and fucking and reported it as an obvious distinguishing feature!?" But nope, just follow through, and I'm willing to bet that for every step in the process the people involved thought it was silly too, but "just followed the system", either because they had no power to do anything else, or out of apathy, or maybe both. Literally no one won with this: the police and criminal justice system wasted time on an obvious wild goose chase, the victim wasn't helped, and presumably more victims will be created and thus even more work for police.
YeGoblynQueenne 3y ago
From the article: “It is circular and dangerous,” Dr. Wells said. “You’ve got a very powerful tool that, if it searches enough faces, will always yield people who look like the person on the surveillance image.” Dr. Wells said the technology compounds an existing problem with eyewitnesses. “They assume when you show them a six-pack, the real person is there,” he said.

by CPLX 3y ago

jtbayly 3y ago

I read it. The article gives a perfect description of how it happened. The software picked her because she looked like the criminal, and then they put her in the lineup. They wouldn’t have had a photo lineup without the software. That’s the whole point. They use the fig leaf of the victim ID, to cover the fact that it’s just facial recognition at the core. The article points out how likely somebody is to mis-ID somebody who the computer has already found is a likely match. Sure, blame the police. But for what? For using facial recognition to ID and arrest people!

MostlyStable 3y ago
This is actually a really good point. My understanding of traditional line ups (do not take my word for this, at all) is that cops put someone(s) in the lineup because they have reason to believe that person is a suspect for some reason, and then then the victim is supposed to pick them out from among other random people. If the entire lineup is selected to be people who "look like the suspect", and, in fact, that's why every one of them is there, then the victim picking them seems to be A) much sketchier and B) much less valuable. The proper way to use this tool seems to be 1. Find all the people who look similar to the low quality pic we have of the suspect. 2. Investigate those people to see if they could have done it. 3. Arrest someone who you hopefully now have additional evidence against besides just "they look like a grainy photo of the suspect" Just throwing a bunch of people who look like the suspect at the victim and just arresting whoever the victim picks seems completely insane. But also, like it doesn't have too much to do with "Facial recognition" broadly speaking and more to do with completely idiotic investigative practices.
- jtbayly 3y ago
  I assumed only one of the photos looked like the perp, which changes it quite a bit. Makes it even worse. How is the person supposed to not pick the facial recognized one at that point? At least if all 6 look like the perp, he might have doubt that he can properly ID them.

dundarious 3y ago

Half of the problem with these tools is that software companies have oversold their competence to a wild degree, to the point where it is trusted as much as a DNA test. "The AI said it's her". No further thinking is required, such as thinking about what photo the AI used, whether an 8-months pregnant woman really had sex with this guy before the stick up, etc. Trust the fancy computer. But I also don't think it's too cynical to believe that at this point, almost everybody up and down the chain knows that these tools don't work very well, but what they like is that they are allowed to defer to these tools. It's like how nobody is fired for buying IBM -- nobody is at fault for relying on the algorithm. This applies to many domains, even consumer tech stuff, hiring practices, etc. Algorithms are just a way to have a color by numbers process where employees have no agency and deserve very little pay, and management has no responsibility and cannot be blamed for mistakes.

woodruffw 3y ago
DNA testing is also significantly less reliable than the police would have you and I believe[1]. HN applies ample and eager skepticism to all kinds of scientific and science-adjacent fields; I eagerly await the day that we do the same for forensic "science." [1]: https://www.theatlantic.com/magazine/archive/2016/06/a-reasonable-doubt/480747/ https://www.theatlantic.com/magazine/archive/2016/06/a-reaso...
- dundarious 3y ago
  I haven't read your link yet, but fair point, I am aware there are definitely some issues with DNA testing as practiced by the justice system, but my prior was that when comparing facial recognition to DNA testing, using ideal conditions for both, DNA testing is on far surer footing than facial recognition -- and the ideal conditions for DNA testing are far more attainable than those for facial recognition. For example, I believe the people acquitted based on DNA testing are usually acquitted for good reason, and not just because it makes me feel good, but I'll read your article.

woodruffw 3y ago

I read the story. The fact that the software produced a lineup rather than singling her out is more or less immaterial; what matters is that it was trusted at all (when, as TFA mentions, it appears to be particularly unreliable with pictures of black people). More than one party can share responsibility in a situation, and in this situation both the police and the software should have blame identified with them.

mapt 3y ago
The software being trusted to pick out similar faces is a GREAT ADVANCE IN SOLVING CRIMES because it scales the process going on in a photo lineup. But it's only being trusted to pick out similar faces. There will ALWAYS be similar faces. It's doing the same part of the task as a photo lineup does. The point is that there has to be MORE than "This person has a similar face". It's a confirmation step for suspects that already exist, or for whom you can find other strong circumstantial evidence for. You have an unlimited supply of similar faces and you don't get to just choose one of them. The great problems with AI tools mostly arise from the fact that the supposed expert users were never taught what the tools do, or how causal inference works - how to discern reality from the evidence available, how to test hypotheses, how to change your reasoning to be valid in the face of wildly disparate scales. If you don't have at least the beginning of Bayesian probability, and you're feeding large datasets into policing, you have no fucking clue what you're doing and your ignorance is immediately dangerous.
- woodruffw 3y ago
  No crimes were solved here. We can't take a thing that literally did nothing except cost taxpayer money, police time, and stress for a pregnant woman and declare victory for it. > It's doing the same part of the task as a photo lineup does. It's doing something subtly different, and arguably pernicious: a lineup is meant to contain a variety of individuals of varying similarities, so that the person selecting from the lineup are themselves tested in their recollection. We have no evidence that this system is being used in this way; instead, it seems to be finding the closest match in an existing dataset under the assumption that anybody who's ever been photographed by the police is permanently a criminal suspect.

arp242 3y ago

I just can't get over how being 8 months pregnant (7 and half months at the time of the crime) was not considered an alibi, or at least considered. "Wait, the person who reported the crime didn't mention that, surely he would have noticed while they were drinking and fucking and reported it as an obvious distinguishing feature!?" But nope, just follow through, and I'm willing to bet that for every step in the process the people involved thought it was silly too, but "just followed the system", either because they had no power to do anything else, or out of apathy, or maybe both. Literally no one won with this: the police and criminal justice system wasted time on an obvious wild goose chase, the victim wasn't helped, and presumably more victims will be created and thus even more work for police.

YeGoblynQueenne 3y ago

From the article: “It is circular and dangerous,” Dr. Wells said. “You’ve got a very powerful tool that, if it searches enough faces, will always yield people who look like the person on the surveillance image.” Dr. Wells said the technology compounds an existing problem with eyewitnesses. “They assume when you show them a six-pack, the real person is there,” he said.