8 ms·

Eight months pregnant and arrested after false facial recognition match

109 points by rmason 3y ago

js2 3y ago
Gift link: https://www.nytimes.com/2023/08/06/technology/facial-recognition-false-arrest.html?unlocked_article_code=mkGZLdgy2Ylqc3ZkobJeGl-EuM-eoEt3OrtirzUeDT1ZL-XvcbAWc5S6DkYtDxL3mO1F4FOackyTRUy6sSSEFxRgDJL77HTeBoIIp3C3oTDDXmXAOOCxQuIapxnCcy76ypkPSWRWX0OVe6AFGmCybhrb2LZToDmiACKpLv7WpSXwxUjdg650wdMiH1Xbm5NEz2bHYqQK3W6wNaoUK_78FoFuj6kTqFNDh7m7zltMNNaSKFokKhBHM8GcNjQCxOsyKTAcS8-2ZiupkC_yN1h7J5qqwNTWWPglBiefnNHhex2Sd1J7S13qnhawijJtHERIZASNUhZfnIYcmS_fs1Xc2g8rUA0bIMqFvToYdA&smid=url-share https://www.nytimes.com/2023/08/06/technology/facial-recogni...
- rtsil 3y ago
  Appreciated!
rmason 3y ago
Detroit has had problem after problem with facial recognition software. But arresting a woman eight months pregnant after the victim identified an eight year old photo should never have happened. She was only in the system in the first place because of an expired drivers license.
jonathankoren 3y ago
New Orleans rolled out facial recognition. It led to exactly zero arrests. https://gizmodo.com/facial-recognition-new-orleans-police-no-arrests-1850692101 https://gizmodo.com/facial-recognition-new-orleans-police-no...
- dd82 3y ago
  Is that because NOLA police used recognition as part of the investigation toolbox, not as an auto-arrest rubber stamp?
  - jonathankoren 3y ago
    Somewhere in NOPD, “Gotta pump those numbers up. Those are rookie numbers in this racket.”
  - BLKNSLVR 3y ago
    I think that's the key to this and many other stories. Police turn on tool turn off brain. Outsourcing their thinking to a tool about which they don't understand the implications of the resulting data. Tool says you; arrest.
barbazoo 3y ago
[flagged]
- lacrimacida 3y ago
  Wait, what? Goes to prison for murder? Is this in relation to abortion bans?
voakbasda 3y ago
> The Wayne County prosecutor, Kym Worthy, considers the arrest warrant in Ms. Woodruff’s case to be “appropriate based upon the facts,” according to a statement issued by her office. Facial recognition is not factual. It is explicitly inferential.
- dd82 3y ago
  Realistically, what do you expect the DA to say? As part of the job, admitting fault for anything pre-emptively goes against all the attributes that get you the job in the first place
  - bell-cot 3y ago
    Fantasy scenario: Voters actually notice such "defend any half-wit thuggery, so long as the perp's have badges" behavior by elected officials, and reliably punish offenders at the ballot box. In theory, every voter who claims to be pro-women's rights, or pro-civil rights, or liberal, or ... should be eager to make this fantasy come true. In practice... Hahahaha!!! Yeah, right. So - why the huge gulf? Could it be that the great majority of voters claiming such moral cred are just Performative Activists, who don't actually give a crap?
    - jj999 3y ago
      Did you ever vote? Between the very limited number of political leaning options available and the personal morality of the candidates, voters can never vote for a candidate they want but only for the candidate they least don't want.
      - bell-cot 3y ago
        Yes (for quite a few decades now). I've noticed that elections often feature both incumbents and challengers. The latter often attack decisions and statements made by the former - "He voted to raise your taxes!", "Look at [video] what he said about X!", etc. By numerous, and seemingly well-researched accounts, an enormous about of behind-the-scenes political work goes into identifying the very best combinations of incumbent behavior and voter hot buttons, to precision-target those expensive (paid political advertising) attacks. If all the politicians, wanna-be politicians, and their armies of consultants and pollsters, seem to believe that where it counts, in the polling booth, voters don't actually care about a certain issue...might all those amply-motivated experts possibly be correct?
jacquesm 3y ago
How did I know she was non-white before clicking the link?
- rtsil 3y ago
  She wasn't the only one either: > Ms. Woodruff is the sixth person to report being falsely accused of a crime as a result of facial recognition technology used by police to match an unknown offender’s face to a photo in a database. All six people have been Black;
- WarOnPrivacy 3y ago
  > How did I know she was non-white before clicking the link? FacRec's history precedes it. The darker the skin the higher the false positive rate. https://duckduckgo.com/?hps=1&q=facial+recognition+skin+color https://duckduckgo.com/?hps=1&q=facial+recognition+skin+colo... Surveillance abettors in general (and LEO specifically) haven't been overly put-off by that feature.
  - 20after4 3y ago
    > by that feature. I see what you did there. It totally is a feature, not a bug.
- an-allen 3y ago
  Because ML facial recognition training sets are notorious for being biased in this way. How a law enforcement organisation and judge could set bail on a person from a biased ML algorithm alone is asanine.
  - jacquesm 3y ago
    Spot on.
- matrix_overload 3y ago
  [flagged]
LouisSayers 3y ago
To me the bigger issue in this story is that the cops didn't use any common sense. I don't live in the US, but from abroad I can tell how messed up the police are over there. The number of stories of brutality, deaths and upright bullying of every day citizens... It's really messed up, perhaps Americans are desensitised, but I can tell you from the other side of the world - it's nowhere like this in other countries, you guys have a BIG problem with how your cops are selected and trained.
- bell-cot 3y ago
  THIS. Hidebound and self-righteous idiots with badges would still be a massive problem, even if the most advanced tech they had was morse-code telegraphs and steel-tipped fountain pens.
- hysan 3y ago
  As an American, living abroad was one of the biggest eye opening experiences to seeing how messed up our law enforcement system is here. I don’t think it’s as much desensitization (which is part of it) as portraying law enforcement as “can do no wrong” heroes in our media and culture. It really skews public perception and results in a lot of people turning a blind eye… until they are the ones affected.
  - pyuser583 3y ago
    As an American who lived abroad, I feel the opposite. Corruption is incredibly common overseas (yes, Europe too). Privacy laws regarding law enforcement are much worse. There’s a really good video on YouTube called “Never Talk To Police.” It’s a law professor and police officer explaining to law students why they should instruct their clients not to talk to police. The police officer compares American and foreign practices, and emphasizes US police use deception, but foreign police use force. I’m sure bad and good police agencies in America and abroad. The best PD I’ve interacted with is in a mid-sized UK city. The worst I’ve seen is in a major American city.
    - lm28469 3y ago
WarOnPrivacy 3y ago
Freaking Kash Hill. If I was limited to reading 10 columnists forever, she'd be one. Her Maxmind story is campfire-good https://fusion.net/story/287592/internet-mapping-glitch-kansas-farm/ https://fusion.net/story/287592/internet-mapping-glitch-kans...
- dekhn 3y ago
  fusion.net doesn't work for me, but I found https://splinternews.com/how-an-internet-mapping-glitch-turned-a-random-kansas-f-1793856052 https://splinternews.com/how-an-internet-mapping-glitch-turn...
mytailorisrich 3y ago
The issue here is procedural, not facial recognition per se. There ought to be a sensible procedure on how to approach people who have been flagged by facial recognition. Now, I don't know what is the normal procedure in the US when police think they have recognised someone who's wanted. Perhaps they do exactly the same as they did to that woman in any case... At least in countries where everyone carries an ID card police could ask for ID first or the person could show their ID to at least and hopefully keep the police in "not sure, let's go easy" mode.
- jacquesm 3y ago
  > There ought to be a sensible procedure on how to approach people who have been flagged by facial recognition. I would suggest 'not'.
- rootusrootus 3y ago
  > Perhaps they do exactly the same as they did to that woman in any case.. It's arguable than in most instances they don't overreact and throw pregnant women into jail for half a day. In a country of 330,000,000 people, you would expect to see stories like this all day, every day, if it were even a little bit routine. The fact that this is national news gives some idea how out of the ordinary it is.
  - elliekelly 3y ago
    Or she’s one of the few with the resources & social connections to get this on the country’s radar.
  - woodruffw 3y ago
    This is a dangerous abductive leap: the victim here is one of six people who were incorrectly arrested due to faulty facial recognition, in just a single mid-sized city. Not all of them received national news stories, and almost certainly never will. We're reading (rightfully) about this woman's story because it has all of the "right" parts: her innocence is painfully obvious, she's pregnant, a professional, a working mother of two, etc. There are others just like her, and the evidence we have points to there being many others just like her; they deserve attention as well, and not to be erased as statistical anomalies.
an-allen 3y ago
Wow this is infuriating. There are so many people in this supply chain that I feel bear moral responsibility for this. From the engineers who have built the ML model for use in this way, to the engineers who designed and built the application to be used in this way, to the people who procured and implemented it, to the public servants who use it on a daily basis when it clearly makes atrocious, avoidable mistakes. To put a woman on bail from a machine learning algorithm output alone is frankly disgusting and reeks of laziness and a lack of care or concern from public servants. I am appalled.
- spicybright 3y ago
  Imagine if software engineers became liable for these sorts of errors. Just like how professional engineers are when they put their stamp on a project.
  - intelVISA 3y ago
    Don't see why not, something like that is the only way (imo) to slow this technological race to the bottom.
  - an-allen 3y ago
    As a software engineer, I think it has to be done. The heres my platform - I cant control who or how folks use it, and have no responsibility of that mentality creates harm… lots of it.
  - rightbyte 3y ago
    The user should have an engineer put a stamp on it if they need a scapegoat for their bad policies. Liability for devs will just entrench Oracle, IBM and whatever.
- TimTheTinker 3y ago
  I don't see any problem with the tool itself; the problem is that it's being used incorrectly and with false assumptions. A facial recognition match should just be the start of one possible line of inquiry, and nothing more. I think the real issue is that the US justice system rewards and penalizes based on the number of convictions achieved. Low = bad, high = good. But high convictions does not equate to promoting justice or domestic tranquility.
dclowd9901 3y ago
> got the surveillance video from the BP gas station It’s obviously not the worst part of this story, but how on earth is anyone supposed to get actual, reliable facial identification from security cameras? Those things have hilariously low resolution on a _good_ day.
CPLX 3y ago
Did anyone read the story? The victim physically picked the woman out of a lineup of mug shot photos placed in front of him. The software placed the woman’s photo in the lineup but the victim of the crime literally ID’d her. Wrongly, of course, but seem like a pretty crucial detail. This is more of an old fashioned police are idiots story than anything else. Like how about investigating instead of just arresting her. Like was she near the crime, did she have an alibi? Etc.
- jtbayly 3y ago
  I read it. The article gives a perfect description of how it happened. The software picked her because she looked like the criminal, and then they put her in the lineup. They wouldn’t have had a photo lineup without the software. That’s the whole point. They use the fig leaf of the victim ID, to cover the fact that it’s just facial recognition at the core. The article points out how likely somebody is to mis-ID somebody who the computer has already found is a likely match. Sure, blame the police. But for what? For using facial recognition to ID and arrest people!
  - MostlyStable 3y ago
    This is actually a really good point. My understanding of traditional line ups (do not take my word for this, at all) is that cops put someone(s) in the lineup because they have reason to believe that person is a suspect for some reason, and then then the victim is supposed to pick them out from among other random people. If the entire lineup is selected to be people who "look like the suspect", and, in fact, that's why every one of them is there, then the victim picking them seems to be A) much sketchier and B) much less valuable. The proper way to use this tool seems to be 1. Find all the people who look similar to the low quality pic we have of the suspect. 2. Investigate those people to see if they could have done it. 3. Arrest someone who you hopefully now have additional evidence against besides just "they look like a grainy photo of the suspect" Just throwing a bunch of people who look like the suspect at the victim and just arresting whoever the victim picks seems completely insane. But also, like it doesn't have too much to do with "Facial recognition" broadly speaking and more to do with completely idiotic investigative practices.

109 points by rmason 3y ago

js2 3y ago

Gift link: https://www.nytimes.com/2023/08/06/technology/facial-recognition-false-arrest.html?unlocked_article_code=mkGZLdgy2Ylqc3ZkobJeGl-EuM-eoEt3OrtirzUeDT1ZL-XvcbAWc5S6DkYtDxL3mO1F4FOackyTRUy6sSSEFxRgDJL77HTeBoIIp3C3oTDDXmXAOOCxQuIapxnCcy76ypkPSWRWX0OVe6AFGmCybhrb2LZToDmiACKpLv7WpSXwxUjdg650wdMiH1Xbm5NEz2bHYqQK3W6wNaoUK_78FoFuj6kTqFNDh7m7zltMNNaSKFokKhBHM8GcNjQCxOsyKTAcS8-2ZiupkC_yN1h7J5qqwNTWWPglBiefnNHhex2Sd1J7S13qnhawijJtHERIZASNUhZfnIYcmS_fs1Xc2g8rUA0bIMqFvToYdA&smid=url-share https://www.nytimes.com/2023/08/06/technology/facial-recogni...

rtsil 3y ago
Appreciated!

rmason 3y ago

Detroit has had problem after problem with facial recognition software. But arresting a woman eight months pregnant after the victim identified an eight year old photo should never have happened. She was only in the system in the first place because of an expired drivers license.

jonathankoren 3y ago

New Orleans rolled out facial recognition. It led to exactly zero arrests. https://gizmodo.com/facial-recognition-new-orleans-police-no-arrests-1850692101 https://gizmodo.com/facial-recognition-new-orleans-police-no...

dd82 3y ago
Is that because NOLA police used recognition as part of the investigation toolbox, not as an auto-arrest rubber stamp?
- jonathankoren 3y ago
  Somewhere in NOPD, “Gotta pump those numbers up. Those are rookie numbers in this racket.”
- BLKNSLVR 3y ago
  I think that's the key to this and many other stories. Police turn on tool turn off brain. Outsourcing their thinking to a tool about which they don't understand the implications of the resulting data. Tool says you; arrest.

barbazoo 3y ago

[flagged]

lacrimacida 3y ago
Wait, what? Goes to prison for murder? Is this in relation to abortion bans?

voakbasda 3y ago

> The Wayne County prosecutor, Kym Worthy, considers the arrest warrant in Ms. Woodruff’s case to be “appropriate based upon the facts,” according to a statement issued by her office. Facial recognition is not factual. It is explicitly inferential.

dd82 3y ago
Realistically, what do you expect the DA to say? As part of the job, admitting fault for anything pre-emptively goes against all the attributes that get you the job in the first place
- bell-cot 3y ago
  Fantasy scenario: Voters actually notice such "defend any half-wit thuggery, so long as the perp's have badges" behavior by elected officials, and reliably punish offenders at the ballot box. In theory, every voter who claims to be pro-women's rights, or pro-civil rights, or liberal, or ... should be eager to make this fantasy come true. In practice... Hahahaha!!! Yeah, right. So - why the huge gulf? Could it be that the great majority of voters claiming such moral cred are just Performative Activists, who don't actually give a crap?
  - jj999 3y ago
    Did you ever vote? Between the very limited number of political leaning options available and the personal morality of the candidates, voters can never vote for a candidate they want but only for the candidate they least don't want.
    - bell-cot 3y ago
      Yes (for quite a few decades now). I've noticed that elections often feature both incumbents and challengers. The latter often attack decisions and statements made by the former - "He voted to raise your taxes!", "Look at [video] what he said about X!", etc. By numerous, and seemingly well-researched accounts, an enormous about of behind-the-scenes political work goes into identifying the very best combinations of incumbent behavior and voter hot buttons, to precision-target those expensive (paid political advertising) attacks. If all the politicians, wanna-be politicians, and their armies of consultants and pollsters, seem to believe that where it counts, in the polling booth, voters don't actually care about a certain issue...might all those amply-motivated experts possibly be correct?

jacquesm 3y ago

How did I know she was non-white before clicking the link?

rtsil 3y ago
She wasn't the only one either: > Ms. Woodruff is the sixth person to report being falsely accused of a crime as a result of facial recognition technology used by police to match an unknown offender’s face to a photo in a database. All six people have been Black;
WarOnPrivacy 3y ago
> How did I know she was non-white before clicking the link? FacRec's history precedes it. The darker the skin the higher the false positive rate. https://duckduckgo.com/?hps=1&q=facial+recognition+skin+color https://duckduckgo.com/?hps=1&q=facial+recognition+skin+colo... Surveillance abettors in general (and LEO specifically) haven't been overly put-off by that feature.
- 20after4 3y ago
  > by that feature. I see what you did there. It totally is a feature, not a bug.
an-allen 3y ago
Because ML facial recognition training sets are notorious for being biased in this way. How a law enforcement organisation and judge could set bail on a person from a biased ML algorithm alone is asanine.
- jacquesm 3y ago
  Spot on.
matrix_overload 3y ago
[flagged]

LouisSayers 3y ago

To me the bigger issue in this story is that the cops didn't use any common sense. I don't live in the US, but from abroad I can tell how messed up the police are over there. The number of stories of brutality, deaths and upright bullying of every day citizens... It's really messed up, perhaps Americans are desensitised, but I can tell you from the other side of the world - it's nowhere like this in other countries, you guys have a BIG problem with how your cops are selected and trained.

bell-cot 3y ago
THIS. Hidebound and self-righteous idiots with badges would still be a massive problem, even if the most advanced tech they had was morse-code telegraphs and steel-tipped fountain pens.
hysan 3y ago
As an American, living abroad was one of the biggest eye opening experiences to seeing how messed up our law enforcement system is here. I don’t think it’s as much desensitization (which is part of it) as portraying law enforcement as “can do no wrong” heroes in our media and culture. It really skews public perception and results in a lot of people turning a blind eye… until they are the ones affected.
- pyuser583 3y ago
  As an American who lived abroad, I feel the opposite. Corruption is incredibly common overseas (yes, Europe too). Privacy laws regarding law enforcement are much worse. There’s a really good video on YouTube called “Never Talk To Police.” It’s a law professor and police officer explaining to law students why they should instruct their clients not to talk to police. The police officer compares American and foreign practices, and emphasizes US police use deception, but foreign police use force. I’m sure bad and good police agencies in America and abroad. The best PD I’ve interacted with is in a mid-sized UK city. The worst I’ve seen is in a major American city.
  - lm28469 3y ago

WarOnPrivacy 3y ago

Freaking Kash Hill. If I was limited to reading 10 columnists forever, she'd be one. Her Maxmind story is campfire-good https://fusion.net/story/287592/internet-mapping-glitch-kansas-farm/ https://fusion.net/story/287592/internet-mapping-glitch-kans...

dekhn 3y ago
fusion.net doesn't work for me, but I found https://splinternews.com/how-an-internet-mapping-glitch-turned-a-random-kansas-f-1793856052 https://splinternews.com/how-an-internet-mapping-glitch-turn...

mytailorisrich 3y ago

The issue here is procedural, not facial recognition per se. There ought to be a sensible procedure on how to approach people who have been flagged by facial recognition. Now, I don't know what is the normal procedure in the US when police think they have recognised someone who's wanted. Perhaps they do exactly the same as they did to that woman in any case... At least in countries where everyone carries an ID card police could ask for ID first or the person could show their ID to at least and hopefully keep the police in "not sure, let's go easy" mode.

jacquesm 3y ago
> There ought to be a sensible procedure on how to approach people who have been flagged by facial recognition. I would suggest 'not'.
rootusrootus 3y ago
> Perhaps they do exactly the same as they did to that woman in any case.. It's arguable than in most instances they don't overreact and throw pregnant women into jail for half a day. In a country of 330,000,000 people, you would expect to see stories like this all day, every day, if it were even a little bit routine. The fact that this is national news gives some idea how out of the ordinary it is.
- elliekelly 3y ago
  Or she’s one of the few with the resources & social connections to get this on the country’s radar.
- woodruffw 3y ago
  This is a dangerous abductive leap: the victim here is one of six people who were incorrectly arrested due to faulty facial recognition, in just a single mid-sized city. Not all of them received national news stories, and almost certainly never will. We're reading (rightfully) about this woman's story because it has all of the "right" parts: her innocence is painfully obvious, she's pregnant, a professional, a working mother of two, etc. There are others just like her, and the evidence we have points to there being many others just like her; they deserve attention as well, and not to be erased as statistical anomalies.

an-allen 3y ago

Wow this is infuriating. There are so many people in this supply chain that I feel bear moral responsibility for this. From the engineers who have built the ML model for use in this way, to the engineers who designed and built the application to be used in this way, to the people who procured and implemented it, to the public servants who use it on a daily basis when it clearly makes atrocious, avoidable mistakes. To put a woman on bail from a machine learning algorithm output alone is frankly disgusting and reeks of laziness and a lack of care or concern from public servants. I am appalled.

spicybright 3y ago
Imagine if software engineers became liable for these sorts of errors. Just like how professional engineers are when they put their stamp on a project.
- intelVISA 3y ago
  Don't see why not, something like that is the only way (imo) to slow this technological race to the bottom.
- an-allen 3y ago
  As a software engineer, I think it has to be done. The heres my platform - I cant control who or how folks use it, and have no responsibility of that mentality creates harm… lots of it.
- rightbyte 3y ago
  The user should have an engineer put a stamp on it if they need a scapegoat for their bad policies. Liability for devs will just entrench Oracle, IBM and whatever.
TimTheTinker 3y ago
I don't see any problem with the tool itself; the problem is that it's being used incorrectly and with false assumptions. A facial recognition match should just be the start of one possible line of inquiry, and nothing more. I think the real issue is that the US justice system rewards and penalizes based on the number of convictions achieved. Low = bad, high = good. But high convictions does not equate to promoting justice or domestic tranquility.

dclowd9901 3y ago

> got the surveillance video from the BP gas station It’s obviously not the worst part of this story, but how on earth is anyone supposed to get actual, reliable facial identification from security cameras? Those things have hilariously low resolution on a _good_ day.

CPLX 3y ago

Did anyone read the story? The victim physically picked the woman out of a lineup of mug shot photos placed in front of him. The software placed the woman’s photo in the lineup but the victim of the crime literally ID’d her. Wrongly, of course, but seem like a pretty crucial detail. This is more of an old fashioned police are idiots story than anything else. Like how about investigating instead of just arresting her. Like was she near the crime, did she have an alibi? Etc.

jtbayly 3y ago
I read it. The article gives a perfect description of how it happened. The software picked her because she looked like the criminal, and then they put her in the lineup. They wouldn’t have had a photo lineup without the software. That’s the whole point. They use the fig leaf of the victim ID, to cover the fact that it’s just facial recognition at the core. The article points out how likely somebody is to mis-ID somebody who the computer has already found is a likely match. Sure, blame the police. But for what? For using facial recognition to ID and arrest people!
- MostlyStable 3y ago
  This is actually a really good point. My understanding of traditional line ups (do not take my word for this, at all) is that cops put someone(s) in the lineup because they have reason to believe that person is a suspect for some reason, and then then the victim is supposed to pick them out from among other random people. If the entire lineup is selected to be people who "look like the suspect", and, in fact, that's why every one of them is there, then the victim picking them seems to be A) much sketchier and B) much less valuable. The proper way to use this tool seems to be 1. Find all the people who look similar to the low quality pic we have of the suspect. 2. Investigate those people to see if they could have done it. 3. Arrest someone who you hopefully now have additional evidence against besides just "they look like a grainy photo of the suspect" Just throwing a bunch of people who look like the suspect at the victim and just arresting whoever the victim picks seems completely insane. But also, like it doesn't have too much to do with "Facial recognition" broadly speaking and more to do with completely idiotic investigative practices.