12 ms·

Why facial recognition has led to false arrests

121 points by dnetesn 6y ago

tyingq 6y ago
"These contain some criminal mugshots, but the bulk of the images comes from non-criminal sources such as passports and state driver’s license compilations; that is, the databases mostly expose ordinary, generally innocent citizens to criminal investigation." Worth noting that a mugshot does not imply criminality. As the lead off in the article shows, lots of innocent people get arrested and thus, have mugshots on file.
- boomboomsubban 6y ago
  Fairly banal crimes can also lead to mugshots. I had to have one taken over a parking violation.
deleted 6y ago
[deleted]
deepstack 6y ago
On hacker news, can we please submit https link instead of http. An S goes a long way in this hostile internet environment now days.
cultus 6y ago
I'm not trying to be snarky here, but can anybody point to an example of machine learning used in law enforcement that hasn't lead to inequitable outcomes?
- hammock 6y ago
  Equally non snarky, what is something in law enforcement that has led to equitable outcomes?
  - glenda 6y ago
    The more I look into the history and tactics of modern policing the more this becomes apparent. Police are willing to use broken machine learning technology because it “works for them” - they don’t necessarily care about accuracy they just want someone locked up.
    - vorpalhex 6y ago
      The police are fundamentally a low paid blue collar labor force with union protections. They are not your college professor or the protagonist of a murder mystery, but more like a plumber or construction worker. The moment you understand that, it's not a mystery that they sort of blindly accept tools that seem to work without deep introspection. They are not equipped to be a data analytics shop, they are equipped to fill out paperwork and sometimes get into tussles.
      - bkor 6y ago
        > The police are fundamentally a low paid blue collar labor force with union protections Please keep in mind that this is not true worldwide. In various countries you're only admitted after 1.5-2 years of police-specific education. You might still not be a police officer if they seem you unfit. Also, you don't seem to value plumbers or construction workers. A colleague used to be a construction worker. It's very interesting to hear the knowledge he has on that. Plus pretty damn handy for understanding how to improve your house. Just because he did a lot of work with his hands doesn't mean he isn't damn bright.
        vorpalhex 6y ago
        At no point did I indicate I don't value blue collar workers such as plumbers or construction workers, quite the opposite. At most I said the police were low paid blue collar workers - and in general I do believe most blue collar workers are underpaid. For what it's worth, I do also value our police forces, even with their flaws. But a blue collar profession has a different set of knowledge handling than a white collar one - it's not that one is better or worse, but they are intrinsically different ways of knowing. > Please keep in mind that this is not true worldwide You're correct and my comment is US centric. What I find interesting is that some US cities have seemingly moved farther from a more educated police force, removing requirements around having college degrees and reducing trainings.
- busterarm 6y ago
  https://nypost.com/2019/08/25/how-nypds-facial-recognition-software-ided-subway-rice-cooker-kook/ https://nypost.com/2019/08/25/how-nypds-facial-recognition-s... https://www.wired.com/story/how-facial-recognition-fighting-child-sex-trafficking/ https://www.wired.com/story/how-facial-recognition-fighting-... https://digital.1105media.com/SP/2019/CSS_1904/SI_APR19_SUP_CSLS_701922949.html#p=10 https://digital.1105media.com/SP/2019/CSS_1904/SI_APR19_SUP_... https://morningconsult.com/opinions/touchless-interface-for-hygienic-access-in-health-care-and-travel/ https://morningconsult.com/opinions/touchless-interface-for-... https://www.nist.gov/news-events/news/2018/05/nist-study-shows-face-recognition-experts-perform-better-ai-partner https://www.nist.gov/news-events/news/2018/05/nist-study-sho... https://www.usatoday.com/story/tech/talkingtech/2018/06/29/capital-gazette-gunman-identified-using-facial-recognition-technology/744344002/ https://www.usatoday.com/story/tech/talkingtech/2018/06/29/c... https://docs.house.gov/meetings/GO/GO00/20200115/110380/HHRG-116-GO00-20200115-SD006.pdf https://docs.house.gov/meetings/GO/GO00/20200115/110380/HHRG... https://nypost.com/2019/08/05/facial-recognition-leads-cops-to-alleged-rapist-in-under-24-hours/ https://nypost.com/2019/08/05/facial-recognition-leads-cops-... https://www.nwitimes.com/news/local/illinois/u-s-marshals-arrest-fugitive-in-munster-payday-loan-business/article_9e538765-53a0-52f7-bc8a-34baa5bbdc54.html https://www.nwitimes.com/news/local/illinois/u-s-marshals-ar... https://www.fbi.gov/news/stories/long-time-fugitive-neil-stammer-captured/long-time-fugitive-neil-stammer-captured https://www.fbi.gov/news/stories/long-time-fugitive-neil-sta... https://nakedsecurity.sophos.com/2017/01/20/alleged-child-molester-caught-after-18-years-thanks-to-facial-recognition/ https://nakedsecurity.sophos.com/2017/01/20/alleged-child-mo... https://www.nextgov.com/emerging-tech/2018/11/cbps-facial-biometrics-program-has-caught-26-alleged-imposters/152957/ https://www.nextgov.com/emerging-tech/2018/11/cbps-facial-bi...
  - pessimizer 6y ago
    Could you just pick the clearest cut one, and type a few sentences explaining it? edit: not trying to get you to do my work for me, but for an existence proof people don't commonly list a bunch of things by name without explaining how they apply. They list one thing (or one class of things, or how to construct a thing), and exactly how it applies.
  - jhayward 6y ago
deleted 6y ago
[deleted]
pbhjpbhj 6y ago
>But the next steps, where investigators first confirm the match, then seek more evidence for an arrest, were poorly done and Williams was brought in. He had to spend 30 hours in jail and post a $1,000 bond before he was freed. // Seems more like "why lack of rigour in policing has lead to false arrests". Facial recognition is evidence, not proof. They clearly didn't view the video and compare it to the person's photo if during post-arrest interview they could immediately tell it was the wrong person? Do they also just arrest people based on other 'witness' reports without any effort towards corroboration?
- faitswulff 6y ago
  If the people in question are black, then yes, arrests or worse happen quite regularly without due diligence.
  - pbhjpbhj 6y ago
    Do you have support for shortcutting of police processes based on skin colour? Like, they arresting officer fills out the 'corroborating evidence' part of the paperwork and says 'none, suspect is black' and that's fine? Or, you just suppose that to be true? AIUI certainty of march is much higher for light-skinned people: do officers get data on match certainty? Forgive my suspicion, but flagrant assumption of racism fuels societal division and strong evidence can avoid unnecessary animosities.
    - faitswulff 6y ago
      There are a preponderance of YouTube videos of encounters like this: > In one especially notorious encounter, police in 2009 arrested a distinguished black Harvard professor, Henry Louis Gates Jr., at his own home in Cambridge, Mass., after a white woman called the police to report a possible break-in. https://www.washingtonpost.com/national/public-outrage-legislation-follow-white-womans-call-to-police-about-black-man-in-central-park/2020/05/27/94b219a6-a049-11ea-9590-1858a893bd59_story.html https://www.washingtonpost.com/national/public-outrage-legis... Breonna Taylor was not only shot to death in her sleep, but was also not given critical aid for twenty minutes after they discovered she was injured: > Mr. Walker told investigators that Ms. Taylor coughed and struggled to breathe for at least five minutes after she was shot, according to The Louisville Courier Journal. She received no medical attention for more than 20 minutes after she was struck, The Courier Journal reported, citing dispatch logs. https://www.nytimes.com/article/breonna-taylor-police.html https://www.nytimes.com/article/breonna-taylor-police.html That, I would say, is a "shortcutting" of police procedure. "Avoiding unnecessary animosity" is less important than facing the problem American society has with racism headlong. Given the evidence that we have, where is the data that it is not happening?
supportlocal4h 6y ago
Facial recognition seems a bit like shoe print id. A valuable clue, but not enough to ruin somebody's life over. That is so obvious that anecdotes about false arrest seem like obvious human incompetence.
- xorfish 6y ago
  And it doesn't matter if the facial recognition is done by a human or a machine. They both produce false results on a regular basis.
- raxxorrax 6y ago
  I don't think it is valuable. Especially with falling crime rates, developing such systems is basically bullshit job in a bullshit industry. Exaggerated, but I haven't really seen neither need nor success of any facreg system. Sure, security agencies might be able to supply some cases when it helped to catch a criminal, but the side effects suggest it isn't even slightly worth it.
  - tetromino_ 6y ago
    Falling crime rates? For some categories of crime, sure, but e.g. murder rates this year have risen sharply in major US cities. https://nyti.ms/3gyoVqd https://nyti.ms/3gyoVqd - "At least 336 people have been murdered in Chicago through July 2, according to the Chicago Police Department; because murders typically increase in the summer, the city is on track to match the 778 deaths in 2016, its deadliest year since the mid-1990s. <...> Chicago is not alone. Before the coronavirus hit, homicides were escalating nationwide in early 2020, and although the lockdown brought a pause, they began rising again as the stay-at-home measures were lifted. A national study showed that homicide rates fell in 39 of 64 major cities during April and began creeping up in May." https://www.nytimes.com/2020/08/04/nyregion/nyc-shootings-coronavirus.html https://www.nytimes.com/2020/08/04/nyregion/nyc-shootings-co... - "On Sunday, another 19 people were shot in New York City, one fatally. Through the first seven months of this year, shootings were up 72 percent over the same period last year and murders rose 30 percent, even as reports of other violent crimes like rape, assault and robbery fell."
throwaway4747l 6y ago
I mean, if your algorithm is accurate 99.9% of the time, and you're using it 24/7 in a country with millions of inhabitants, you should expect tens of thousands of false positives every day. This is why some wet blankets have been strangely skeptical of delegating all responsability to The Algorithm.
- Guest42 6y ago
  Having spent time in the industry with one of the supposed “top companies”, the accuracy was only good when the model was over-fit on good training data, real world performance wasn’t close nor was it evaluated much.
- coldcode 6y ago
  If only it was as good as 99.9, it's probably amazing if its 50.
- sevencolors 6y ago
  I love how you just handwave away the possibility of destroying someone's life with a false positive. This is an area like automated driving where the error rate needs to be 0 if it's going to be relied upon as truth.
  - throwaway4747l 6y ago
    I'm actually agreeing with you, perhaps the sarcastic tone didn't go through.
mytailorisrich 6y ago
The issue highlighted in this article is that it seems that police deemed facial recognition as reliable as DNA so that if the computer says it's that person then they are already guilty. This is a simple issue that could be easily solved by making sure that facial recognition should be treated as a very good lead but not as a smoking gun, so that the police approaches "suspects" more neutrally before deciding whether to arrest them.
- teddyh 6y ago
  So how do you get police to behave that way?
BurningFrog 6y ago
Every method of finding criminals result in innocent people being arrested from time to time. That's the nature of arrests. Maybe there is something uniquely bad about facial recognition, but to show that, compare it to other techniques, not to a perfection standard.
- LatteLazy 6y ago
  The open question here is who does this effect. If the result is 1000 wrong arrests out of 10mil, that might be acceptable. But if ever one of those arrests is that 1 poor guy who get's arrested every time he steps out of his house because his face is a universal match, that's a very different result. Ditto if this leads to large numbers of incorrect arrests in certain groups (race, gender, age etc). This is the problem with anything involving people: distributions matter a lot and distributions are hard...
- Jtsummers 6y ago
  There are several issues, though. 1. As discussed in the article, due to training sets or methodology, these systems are poor at distinguishing non-Caucasian faces. Or at least poor relative to a human looking at the image and the person together. 2. Police (though not just them) have an unfounded trust in the machines. 3. Police take this unfounded trust and push through arrests before they realistically should (like in the opening story of the article). 4. People arrested for felonies are fucked in the US. It can take years to get your record expunged. Until then your name shows up in lots of searches, you can lose licenses to conduct some kinds of business (or be unable to obtain them). (Expunging can also be very expensive.) You may not mind innocent people being caught up in the dragnet, but as a society we probably should care a lot more than we do.
  - BurningFrog 6y ago
    Yeah, those issues may be big problems. Or they may not. My point is that the way to know is to compare this tool with others and weigh pros and cons.
NPMaxwell 6y ago
This article appears in my HN feed with an article about throwing away React. The React article explains that having code that prevents engineers from doing whatever they want to is essential. If we can't trust engineers coding, why do we trust police arresting? Restraints are needed.
bluesign 6y ago
Problem here is for arrest they claim to be using not facial recognition but witness identification. In theory it sounds like good. But the problem is when you identify some similar face to suspect then show this with some random faces, you are actually giving a multiple choice to witness, which is biased for the match you have found.
Dominisi 6y ago
I worked in a Law Enforcment office that used Facial Recognition. Every single time we would give officers a potential match, it came with a disclaimer that it was a POTENTIAL match. It was up to the detectives and officers to actually do their job and find other pillars of information. This isn't a failure of facial recognition, its a failure of the stupidest people I went to high school with thinking its some sort of perfect system.
- teachrdan 6y ago
  If the software vendor knows the end user will not use their product as intended, at what point is the vendor complicit in the impacts of that misuse?
- LargeWu 6y ago
  Given the well-documented penchant of police to use “fits the description” as a pretext to harass the public, especially people of color, I would say it is borderline negligent to allow the use of algorithmically generated probable cause justifications.
- michaelmrose 6y ago
  You can't paint a proper moral picture of a situation by imagining your responsibility is compartmentalized and that you operate in an environment where others will or at least could in theory operate ideally. Such an approximation must as part of the moral calculus be incrementally replaced with a more accurate picture that takes into account the actual foibles of your fellow humans actually behave. If given how people actually behave evil or injustice is done that is your fault. Not partially your fault, your fault guilt doesn't divide it just multiplies. You are I think bound not to do the least you can do but rather the most you can do. The best idea I've heard in this thread is to always show multiple results with no indication of which was higher priority. Force a human into the loop.
renewiltord 6y ago
Most people are incompetent. The tax lawyer down the street, the pharmacist, the dental assistant. It's just that most people aren't in a position to hurt others because there are checks: - the recipient checks the pharmacist's work - the dentist checks his assistant's work - the state allows the tax lawyer some leeway 16% of Americans have an IQ less than 85. 50% have an IQ less than 100. If you build a tool that requires a person to have average IQ, it will fail with half of all users. That doesn't mean it's a bad idea: after all, you only need to convince the buyer, not the user.
sdsvsdgggggg 6y ago
As if humans have never misidentified people or arrested the wrong person? I find these articles very silly. Maybe the photo on the drivers license was not good enough, so only in the real world could the police see that it was not the same person. In any case, why not scan the database automatically (the evil, evil facial recognition), and then double check by humans? Even when humans police, I think it is always just about probabilities. Then they follow up (ideally) and try to drive the probability of being correct higher. Obviously nobody should be tried automatically without any recourse.
- Shared404 6y ago
  > why not scan the database automatically (the evil, evil facial recognition), and then double check by humans? Because people tend to think "The computer said this was right, therefore it is unequivocally the correct answer." This is obviously incorrect, but is how many laypeople view it, even if instructed otherwise.
  - tehwebguy 6y ago
    In fact that's exactly what happened. The police realized it was the wrong guy and still brought him in, he still lose 30 hours and had to post bail. I cant even imagine if two morons showed up to my house, saw that I was the wrong person, and then arrested me in front of my family anyway.
    - sdsvsdgggggg 6y ago
      But then it can't possibly be the "people just believe computers" effect, because the article explicitly states the police realized the computer was wrong.
      - Shared404 6y ago
        Then in this case it's the same as phishing emails. People know it's wrong, and yet still follow along anyways. This is not necessarily a forgone conclusion, but occurs to often to dismiss.
magwa101 6y ago
AKA "Why human identification has lead to such a poor criminal justice outcome"
jakobmartz3 6y ago
Those police should have been required to make sure he was a match. Feel like it was mostly due to racism and thats fckin horrible. Then they blame it on the computer. like yeah but its also their fault for not double checking.
grumio 6y ago
What's the incentive to reduce false arrests? Are the arrestors' jobs at stake? Is their wallet at stake? Why put in the extra effort if there are no consequences for being wrong? Most of the time it seems the worst-case scenario for the perpetrator of the false arrest is being relegated to desk-duty or a paid suspension. If there's a settlement it's the tax-payers that foot the bill. Until that changes, there's basically no reason for them to make the effort to reduce false arrests. This isn't a problem that will be solved with recognition algorithms getting a few percentage points more accurate every year. It will require actual reform.
oppositelock 6y ago
The best algorithm that NIST tested had a 0.01% false positive rate and it was uniform across races. Consider an airport like Heathrow, which uses facial recognition widely. 220,000 people travel through heathrow per day. With a 0.01% false positive rate, you are falsely flagging 22 people per day. You can not rely on facial recognition to jail people.
kortex 6y ago
> Williams’ case is a signal to stop the ad hoc adoption of facial recognition before an injustice occurs that cannot be undone.” Arguably, Williams' case is that injustice. It should not have happened in the first place (although anyone in the field could have told you it would be inevitable). Clearly the machine-human interface is insufficent. A big honking warning is still insufficient to overcome automation bias (see other top level comment). Perhaps the "lineup" functionality needs to be baked in. The system won't divulge details until a(n) operator(s) selects hits from a pool.
GuB-42 6y ago
The real question is: overall did face recognition lead to a higher proportion of false arrests, and even more importantly false convictions. Clearly, facial recognition can make mistakes that lead to false arrests. Of course, it is important to minimize that number, that alone doesn't tell us if facial recognition is justified. The thing is: - An arrest is not a conviction, it is important to know (if possible) how much of these facial recognition false arrests led to false convictions. I suspect not a lot, because judges and attorneys are still human. - How much false arrests did facial recognition prevent? It can happen if the system correctly identifies the culprit before an innocent is arrested. It can also be used to cross-check false/unreliable testimony. Note: I am not talking about the privacy implication of facial recognition, only its effect on criminal investigations.

121 points by dnetesn 6y ago

"These contain some criminal mugshots, but the bulk of the images comes from non-criminal sources such as passports and state driver’s license compilations; that is, the databases mostly expose ordinary, generally innocent citizens to criminal investigation." Worth noting that a mugshot does not imply criminality. As the lead off in the article shows, lots of innocent people get arrested and thus, have mugshots on file.

boomboomsubban 6y ago
Fairly banal crimes can also lead to mugshots. I had to have one taken over a parking violation.

deleted 6y ago

[deleted]

deepstack 6y ago

On hacker news, can we please submit https link instead of http. An S goes a long way in this hostile internet environment now days.

cultus 6y ago

I'm not trying to be snarky here, but can anybody point to an example of machine learning used in law enforcement that hasn't lead to inequitable outcomes?

hammock 6y ago
Equally non snarky, what is something in law enforcement that has led to equitable outcomes?
- glenda 6y ago
  The more I look into the history and tactics of modern policing the more this becomes apparent. Police are willing to use broken machine learning technology because it “works for them” - they don’t necessarily care about accuracy they just want someone locked up.
  - vorpalhex 6y ago
    The police are fundamentally a low paid blue collar labor force with union protections. They are not your college professor or the protagonist of a murder mystery, but more like a plumber or construction worker. The moment you understand that, it's not a mystery that they sort of blindly accept tools that seem to work without deep introspection. They are not equipped to be a data analytics shop, they are equipped to fill out paperwork and sometimes get into tussles.
    - bkor 6y ago
      > The police are fundamentally a low paid blue collar labor force with union protections Please keep in mind that this is not true worldwide. In various countries you're only admitted after 1.5-2 years of police-specific education. You might still not be a police officer if they seem you unfit. Also, you don't seem to value plumbers or construction workers. A colleague used to be a construction worker. It's very interesting to hear the knowledge he has on that. Plus pretty damn handy for understanding how to improve your house. Just because he did a lot of work with his hands doesn't mean he isn't damn bright.
      - vorpalhex 6y ago
        At no point did I indicate I don't value blue collar workers such as plumbers or construction workers, quite the opposite. At most I said the police were low paid blue collar workers - and in general I do believe most blue collar workers are underpaid. For what it's worth, I do also value our police forces, even with their flaws. But a blue collar profession has a different set of knowledge handling than a white collar one - it's not that one is better or worse, but they are intrinsically different ways of knowing. > Please keep in mind that this is not true worldwide You're correct and my comment is US centric. What I find interesting is that some US cities have seemingly moved farther from a more educated police force, removing requirements around having college degrees and reducing trainings.
busterarm 6y ago
https://nypost.com/2019/08/25/how-nypds-facial-recognition-software-ided-subway-rice-cooker-kook/ https://nypost.com/2019/08/25/how-nypds-facial-recognition-s... https://www.wired.com/story/how-facial-recognition-fighting-child-sex-trafficking/ https://www.wired.com/story/how-facial-recognition-fighting-... https://digital.1105media.com/SP/2019/CSS_1904/SI_APR19_SUP_CSLS_701922949.html#p=10 https://digital.1105media.com/SP/2019/CSS_1904/SI_APR19_SUP_... https://morningconsult.com/opinions/touchless-interface-for-hygienic-access-in-health-care-and-travel/ https://morningconsult.com/opinions/touchless-interface-for-... https://www.nist.gov/news-events/news/2018/05/nist-study-shows-face-recognition-experts-perform-better-ai-partner https://www.nist.gov/news-events/news/2018/05/nist-study-sho... https://www.usatoday.com/story/tech/talkingtech/2018/06/29/capital-gazette-gunman-identified-using-facial-recognition-technology/744344002/ https://www.usatoday.com/story/tech/talkingtech/2018/06/29/c... https://docs.house.gov/meetings/GO/GO00/20200115/110380/HHRG-116-GO00-20200115-SD006.pdf https://docs.house.gov/meetings/GO/GO00/20200115/110380/HHRG... https://nypost.com/2019/08/05/facial-recognition-leads-cops-to-alleged-rapist-in-under-24-hours/ https://nypost.com/2019/08/05/facial-recognition-leads-cops-... https://www.nwitimes.com/news/local/illinois/u-s-marshals-arrest-fugitive-in-munster-payday-loan-business/article_9e538765-53a0-52f7-bc8a-34baa5bbdc54.html https://www.nwitimes.com/news/local/illinois/u-s-marshals-ar... https://www.fbi.gov/news/stories/long-time-fugitive-neil-stammer-captured/long-time-fugitive-neil-stammer-captured https://www.fbi.gov/news/stories/long-time-fugitive-neil-sta... https://nakedsecurity.sophos.com/2017/01/20/alleged-child-molester-caught-after-18-years-thanks-to-facial-recognition/ https://nakedsecurity.sophos.com/2017/01/20/alleged-child-mo... https://www.nextgov.com/emerging-tech/2018/11/cbps-facial-biometrics-program-has-caught-26-alleged-imposters/152957/ https://www.nextgov.com/emerging-tech/2018/11/cbps-facial-bi...
- pessimizer 6y ago
  Could you just pick the clearest cut one, and type a few sentences explaining it? edit: not trying to get you to do my work for me, but for an existence proof people don't commonly list a bunch of things by name without explaining how they apply. They list one thing (or one class of things, or how to construct a thing), and exactly how it applies.
- jhayward 6y ago

deleted 6y ago

[deleted]

pbhjpbhj 6y ago

>But the next steps, where investigators first confirm the match, then seek more evidence for an arrest, were poorly done and Williams was brought in. He had to spend 30 hours in jail and post a $1,000 bond before he was freed. // Seems more like "why lack of rigour in policing has lead to false arrests". Facial recognition is evidence, not proof. They clearly didn't view the video and compare it to the person's photo if during post-arrest interview they could immediately tell it was the wrong person? Do they also just arrest people based on other 'witness' reports without any effort towards corroboration?

faitswulff 6y ago
If the people in question are black, then yes, arrests or worse happen quite regularly without due diligence.
- pbhjpbhj 6y ago
  Do you have support for shortcutting of police processes based on skin colour? Like, they arresting officer fills out the 'corroborating evidence' part of the paperwork and says 'none, suspect is black' and that's fine? Or, you just suppose that to be true? AIUI certainty of march is much higher for light-skinned people: do officers get data on match certainty? Forgive my suspicion, but flagrant assumption of racism fuels societal division and strong evidence can avoid unnecessary animosities.
  - faitswulff 6y ago
    There are a preponderance of YouTube videos of encounters like this: > In one especially notorious encounter, police in 2009 arrested a distinguished black Harvard professor, Henry Louis Gates Jr., at his own home in Cambridge, Mass., after a white woman called the police to report a possible break-in. https://www.washingtonpost.com/national/public-outrage-legislation-follow-white-womans-call-to-police-about-black-man-in-central-park/2020/05/27/94b219a6-a049-11ea-9590-1858a893bd59_story.html https://www.washingtonpost.com/national/public-outrage-legis... Breonna Taylor was not only shot to death in her sleep, but was also not given critical aid for twenty minutes after they discovered she was injured: > Mr. Walker told investigators that Ms. Taylor coughed and struggled to breathe for at least five minutes after she was shot, according to The Louisville Courier Journal. She received no medical attention for more than 20 minutes after she was struck, The Courier Journal reported, citing dispatch logs. https://www.nytimes.com/article/breonna-taylor-police.html https://www.nytimes.com/article/breonna-taylor-police.html That, I would say, is a "shortcutting" of police procedure. "Avoiding unnecessary animosity" is less important than facing the problem American society has with racism headlong. Given the evidence that we have, where is the data that it is not happening?

supportlocal4h 6y ago

Facial recognition seems a bit like shoe print id. A valuable clue, but not enough to ruin somebody's life over. That is so obvious that anecdotes about false arrest seem like obvious human incompetence.

xorfish 6y ago
And it doesn't matter if the facial recognition is done by a human or a machine. They both produce false results on a regular basis.
raxxorrax 6y ago
I don't think it is valuable. Especially with falling crime rates, developing such systems is basically bullshit job in a bullshit industry. Exaggerated, but I haven't really seen neither need nor success of any facreg system. Sure, security agencies might be able to supply some cases when it helped to catch a criminal, but the side effects suggest it isn't even slightly worth it.
- tetromino_ 6y ago
  Falling crime rates? For some categories of crime, sure, but e.g. murder rates this year have risen sharply in major US cities. https://nyti.ms/3gyoVqd https://nyti.ms/3gyoVqd - "At least 336 people have been murdered in Chicago through July 2, according to the Chicago Police Department; because murders typically increase in the summer, the city is on track to match the 778 deaths in 2016, its deadliest year since the mid-1990s. <...> Chicago is not alone. Before the coronavirus hit, homicides were escalating nationwide in early 2020, and although the lockdown brought a pause, they began rising again as the stay-at-home measures were lifted. A national study showed that homicide rates fell in 39 of 64 major cities during April and began creeping up in May." https://www.nytimes.com/2020/08/04/nyregion/nyc-shootings-coronavirus.html https://www.nytimes.com/2020/08/04/nyregion/nyc-shootings-co... - "On Sunday, another 19 people were shot in New York City, one fatally. Through the first seven months of this year, shootings were up 72 percent over the same period last year and murders rose 30 percent, even as reports of other violent crimes like rape, assault and robbery fell."

throwaway4747l 6y ago

I mean, if your algorithm is accurate 99.9% of the time, and you're using it 24/7 in a country with millions of inhabitants, you should expect tens of thousands of false positives every day. This is why some wet blankets have been strangely skeptical of delegating all responsability to The Algorithm.

Guest42 6y ago
Having spent time in the industry with one of the supposed “top companies”, the accuracy was only good when the model was over-fit on good training data, real world performance wasn’t close nor was it evaluated much.
coldcode 6y ago
If only it was as good as 99.9, it's probably amazing if its 50.
sevencolors 6y ago
I love how you just handwave away the possibility of destroying someone's life with a false positive. This is an area like automated driving where the error rate needs to be 0 if it's going to be relied upon as truth.
- throwaway4747l 6y ago
  I'm actually agreeing with you, perhaps the sarcastic tone didn't go through.

mytailorisrich 6y ago

The issue highlighted in this article is that it seems that police deemed facial recognition as reliable as DNA so that if the computer says it's that person then they are already guilty. This is a simple issue that could be easily solved by making sure that facial recognition should be treated as a very good lead but not as a smoking gun, so that the police approaches "suspects" more neutrally before deciding whether to arrest them.

teddyh 6y ago
So how do you get police to behave that way?

BurningFrog 6y ago

Every method of finding criminals result in innocent people being arrested from time to time. That's the nature of arrests. Maybe there is something uniquely bad about facial recognition, but to show that, compare it to other techniques, not to a perfection standard.

LatteLazy 6y ago
The open question here is who does this effect. If the result is 1000 wrong arrests out of 10mil, that might be acceptable. But if ever one of those arrests is that 1 poor guy who get's arrested every time he steps out of his house because his face is a universal match, that's a very different result. Ditto if this leads to large numbers of incorrect arrests in certain groups (race, gender, age etc). This is the problem with anything involving people: distributions matter a lot and distributions are hard...
Jtsummers 6y ago
There are several issues, though. 1. As discussed in the article, due to training sets or methodology, these systems are poor at distinguishing non-Caucasian faces. Or at least poor relative to a human looking at the image and the person together. 2. Police (though not just them) have an unfounded trust in the machines. 3. Police take this unfounded trust and push through arrests before they realistically should (like in the opening story of the article). 4. People arrested for felonies are fucked in the US. It can take years to get your record expunged. Until then your name shows up in lots of searches, you can lose licenses to conduct some kinds of business (or be unable to obtain them). (Expunging can also be very expensive.) You may not mind innocent people being caught up in the dragnet, but as a society we probably should care a lot more than we do.
- BurningFrog 6y ago
  Yeah, those issues may be big problems. Or they may not. My point is that the way to know is to compare this tool with others and weigh pros and cons.

NPMaxwell 6y ago

This article appears in my HN feed with an article about throwing away React. The React article explains that having code that prevents engineers from doing whatever they want to is essential. If we can't trust engineers coding, why do we trust police arresting? Restraints are needed.

bluesign 6y ago

Problem here is for arrest they claim to be using not facial recognition but witness identification. In theory it sounds like good. But the problem is when you identify some similar face to suspect then show this with some random faces, you are actually giving a multiple choice to witness, which is biased for the match you have found.

Dominisi 6y ago

I worked in a Law Enforcment office that used Facial Recognition. Every single time we would give officers a potential match, it came with a disclaimer that it was a POTENTIAL match. It was up to the detectives and officers to actually do their job and find other pillars of information. This isn't a failure of facial recognition, its a failure of the stupidest people I went to high school with thinking its some sort of perfect system.

teachrdan 6y ago
If the software vendor knows the end user will not use their product as intended, at what point is the vendor complicit in the impacts of that misuse?
LargeWu 6y ago
Given the well-documented penchant of police to use “fits the description” as a pretext to harass the public, especially people of color, I would say it is borderline negligent to allow the use of algorithmically generated probable cause justifications.
michaelmrose 6y ago
You can't paint a proper moral picture of a situation by imagining your responsibility is compartmentalized and that you operate in an environment where others will or at least could in theory operate ideally. Such an approximation must as part of the moral calculus be incrementally replaced with a more accurate picture that takes into account the actual foibles of your fellow humans actually behave. If given how people actually behave evil or injustice is done that is your fault. Not partially your fault, your fault guilt doesn't divide it just multiplies. You are I think bound not to do the least you can do but rather the most you can do. The best idea I've heard in this thread is to always show multiple results with no indication of which was higher priority. Force a human into the loop.

renewiltord 6y ago

Most people are incompetent. The tax lawyer down the street, the pharmacist, the dental assistant. It's just that most people aren't in a position to hurt others because there are checks: - the recipient checks the pharmacist's work - the dentist checks his assistant's work - the state allows the tax lawyer some leeway 16% of Americans have an IQ less than 85. 50% have an IQ less than 100. If you build a tool that requires a person to have average IQ, it will fail with half of all users. That doesn't mean it's a bad idea: after all, you only need to convince the buyer, not the user.

sdsvsdgggggg 6y ago

As if humans have never misidentified people or arrested the wrong person? I find these articles very silly. Maybe the photo on the drivers license was not good enough, so only in the real world could the police see that it was not the same person. In any case, why not scan the database automatically (the evil, evil facial recognition), and then double check by humans? Even when humans police, I think it is always just about probabilities. Then they follow up (ideally) and try to drive the probability of being correct higher. Obviously nobody should be tried automatically without any recourse.

Shared404 6y ago
> why not scan the database automatically (the evil, evil facial recognition), and then double check by humans? Because people tend to think "The computer said this was right, therefore it is unequivocally the correct answer." This is obviously incorrect, but is how many laypeople view it, even if instructed otherwise.
- tehwebguy 6y ago
  In fact that's exactly what happened. The police realized it was the wrong guy and still brought him in, he still lose 30 hours and had to post bail. I cant even imagine if two morons showed up to my house, saw that I was the wrong person, and then arrested me in front of my family anyway.
  - sdsvsdgggggg 6y ago
    But then it can't possibly be the "people just believe computers" effect, because the article explicitly states the police realized the computer was wrong.
    - Shared404 6y ago
      Then in this case it's the same as phishing emails. People know it's wrong, and yet still follow along anyways. This is not necessarily a forgone conclusion, but occurs to often to dismiss.

magwa101 6y ago

AKA "Why human identification has lead to such a poor criminal justice outcome"

jakobmartz3 6y ago

Those police should have been required to make sure he was a match. Feel like it was mostly due to racism and thats fckin horrible. Then they blame it on the computer. like yeah but its also their fault for not double checking.

grumio 6y ago

What's the incentive to reduce false arrests? Are the arrestors' jobs at stake? Is their wallet at stake? Why put in the extra effort if there are no consequences for being wrong? Most of the time it seems the worst-case scenario for the perpetrator of the false arrest is being relegated to desk-duty or a paid suspension. If there's a settlement it's the tax-payers that foot the bill. Until that changes, there's basically no reason for them to make the effort to reduce false arrests. This isn't a problem that will be solved with recognition algorithms getting a few percentage points more accurate every year. It will require actual reform.

oppositelock 6y ago

The best algorithm that NIST tested had a 0.01% false positive rate and it was uniform across races. Consider an airport like Heathrow, which uses facial recognition widely. 220,000 people travel through heathrow per day. With a 0.01% false positive rate, you are falsely flagging 22 people per day. You can not rely on facial recognition to jail people.

kortex 6y ago

> Williams’ case is a signal to stop the ad hoc adoption of facial recognition before an injustice occurs that cannot be undone.” Arguably, Williams' case is that injustice. It should not have happened in the first place (although anyone in the field could have told you it would be inevitable). Clearly the machine-human interface is insufficent. A big honking warning is still insufficient to overcome automation bias (see other top level comment). Perhaps the "lineup" functionality needs to be baked in. The system won't divulge details until a(n) operator(s) selects hits from a pool.

GuB-42 6y ago

The real question is: overall did face recognition lead to a higher proportion of false arrests, and even more importantly false convictions. Clearly, facial recognition can make mistakes that lead to false arrests. Of course, it is important to minimize that number, that alone doesn't tell us if facial recognition is justified. The thing is: - An arrest is not a conviction, it is important to know (if possible) how much of these facial recognition false arrests led to false convictions. I suspect not a lot, because judges and attorneys are still human. - How much false arrests did facial recognition prevent? It can happen if the system correctly identifies the culprit before an innocent is arrested. It can also be used to cross-check false/unreliable testimony. Note: I am not talking about the privacy implication of facial recognition, only its effect on criminal investigations.