27 ms·

He spent 10 days in jail after facial recognition led to arrest of the wrong man

259 points by sharkweek 6y ago

mikestew 6y ago
I wish I could say that I was shocked that the victim of bad facial recognition is black. I wish I could say that I'm shocked that authorities take the word of software over looking at the evidence and thinking, "umm, doesn't look like him..." Or perhaps someone did look and said, "meh, they all look the same to me." All I know is that I'm highly skeptical that this white guy would suffer the same fate, even if I were to have a similar criminal record. Facial recognition seems to be the polygraph for a new century. But unlike a polygraph, it mostly false positives only for dark-skinned people. While I'm not quite ready to allow my cynicism to let me think it is an intentional feature, I question how many are demanding a fix.
- AmVess 6y ago
  The last thing police need are more tools that don't work and they don't understand how to use. That they still use polygraphs should make everyone's hair stand on end.
  - tartoran 6y ago
    As I understand polygraphs are used like placebos/intimidation tactic to get the suspects to spill the beans. As far as I know the polygraph isn't accepted as evidence in court unless all parties agree to it. However, the dirtiest thing is to use all kind useless evidence and intimidation techniques to scare otherwise innocent people into taking plea deals. This whole guilty plea saga is a disgrace to the justice system.
    - jonny_eh 6y ago
      > As far as I know the polygraph isn't accepted as evidence in court unless all parties agree to it That's not the only place where they can be used, and cause damage.
    - _underfl0w_ 6y ago
      Correct. It's widely considered inadmissible as evidence in court. They're also rarely used outside of the U.S. AFAIK. Recordings of interrogations that feature them also show that they tend to come along with other appeals to ethos ("this is doctor so-and-so...") and hand-wavey forms of intimidation like touting their "seriousness" or "efficacy". Anything that requires a person to "interpret" its output is likely not very deterministic, and so likely not very reliable for establishing... anything. This is akin to palm reading or graphology. Unfortunately, revealing to the police in such a situation exactly how foolish they look by getting behind such a device isn't exactly advisable either!
      - amiga 6y ago
        The Penn and Teller series, Bullshit, features an episode on polygraph machines and interviewer techniques. It's a good watch.
    - KittenInABox 6y ago
      A failed polygraph while inadmissible in court will still be used by police during their own investigations to confirm or deny their "gut feeling".
      - AmVess 6y ago
        I failed a polygraph. There was a crime at a store, and my name came up when they searched through the ATM records. Police had ample evidence that I was in no way related to the crime, but kept trying to find something to pin on me because of the failed test. Those particular detectives had complete faith in the lie detector as a valid tool of police work. Video evidence of me entering the store, using the atm, and immediately leaving the store? That was not evidence to them because I failed the polygraph. I take an exceedingly dim view of any police department that uses a polygraph in any capacity. They've been proven time and time and time again to be worthless as a tool, but police still use them. The rough equivalent is using tarot cards as a tool for solving crimes.
        CraneWorm 6y ago
        You assume they wanted truth when in reality they only want confession.
        syshum 6y ago
        There are many things in law enforcement that have either been proven to be worthless, near worthless or never been proven to be valid... Eye Witness Testimony for example, aside from the fact the people are terrible at recall anyway and High Stress causes memory problems above and beyond the normal memory issues of the human mind, it is very very very easy to manipulate a victim either directly or indirectly to identify the "target" no necessarily the actual criminal police "drug" dogs are another one, it is more or less a blank check to probable cause for a search. Then there is one people never even question because it is the foundation of the legal system, the good old finger print... It always amazes me that with DNA it is always probabilistic, "1 in 4 billion people will have this same DNA" but with Finger Prints is an absolute, Match no match.. Call me crazy but I don't think that position is scientifically valid
    - throwaway201103 6y ago
      > the dirtiest thing is to use all kind useless evidence and intimidation techniques to scare otherwise innocent people into taking plea deals They don't even need to do that. They can just lie to you. "Hey, your buddy just totally ratted you out in the other room, you better tell us your side of it if you don't want to take the rap for everything"
      - SerLava 6y ago
        Even worse, they can pretend they have evidence that they don't, pretend they believe the evidence, and pretend it's enough to convict you by itself. They can tell you to just say you were there, and they'll go easy on you. You lie and say you were there, now they have evidence that can actually lock you up, even if you are a random person off the street.
        bluejekyll 6y ago
        To a degree, but I also think this will be changing over time as more and more innocent people are found to have been convicted on false statements used through coercive lying tactics by the police. For example, in this case police threatened to arrest the mans wife if he didn’t confess to killing his child (a new trial was ordered): https://brooklyneagle.com/articles/2014/02/20/police-can-lie-to-suspects-during-interrogations-says-court-of-appeals/ https://brooklyneagle.com/articles/2014/02/20/police-can-lie... But they went on to say it was still acceptable to lie to suspects... I imagine this murky area of the law will eventually say that police cannot lie to suspects. The Central Park Five is mentioned in that article and is a blatant example of police lying to coerce testimony and then putting innocent children in prison.
        syshum 6y ago
        Even Worse than all of that, they can use all of those things, while threatening you with dozens of charges and decades of imprisonment then say "but if you act right now and sign this we will only give you a couple of years and then you will be free" in any other context these tactics themselves would be illegal, and i am firm believer that you can not enforce the law by breaking, and that the police should NEVER how the power to do things the public can not
  - pmiller2 6y ago
    See also "touch DNA": https://www.techdirt.com/articles/20191026/12493943266/federal-court-says-touch-dna-analysis-is-mostly-guesswork-that-cant-be-used-as-evidence.shtml https://www.techdirt.com/articles/20191026/12493943266/feder...
- gwern 6y ago
  Are we looking at the same photos? They look uncannily alike to me, to the point where I'm left wondering if the forged license actually reuses a photo from Parks. (As a double convict in NJ, I suspect his earlier mugshots would not be hard for the forger to get. Certainly no harder than forging a license.)
  - crooked-v 6y ago
    "Uncannily"? I don't see it. Different mouth proportions compared to the nose, different cheekbones, different eyebrow lengths...
  - sebastien_b 6y ago
    They do look similar (enough to become a potential suspect, though I don't think should be enough for a warrant), and given that police departments don't "protect" their mugshots, I wouldn't be surprised if the picture was indeed reused (or photoshopped). But I really wonder if the same officers that dealt with the suspect also questioned the innocent victim in this case - it seems like this wouldn't be the case (and gross negligence on their part) - those people should've been able to tell them apart (instead of solely relying on the picture they got from the license). I think this is what the judge wanted instead of just the facial recognition "evidence" (which eventually led to the charges being dropped) - something like, oh I don't know, the fingerprints from the abandoned vehicle? Basic investigative work, perhaps?
    - pmiller2 6y ago
      Eh, I wouldn't even say that. I'd say they look similar enough to be put into the same photo lineup, but I wouldn't even drag the guy from the article in to do a physical lineup, much less arrest him.
      - sebastien_b 6y ago
        Agree that a photo alone shouldn’t be enough for an arrest warrant. Like I said - either the officers that dealt with the suspect didn’t deal with/look at the victim, or else they’re blind (or trying to meet a quota).
- cgriswald 6y ago
  > All I know is that I'm highly skeptical that this white guy would suffer the same fate, even if I were to have a similar criminal record. It is not clear to me that a prosecutor's office which has demonstrated that it is not interested in justice, would have cared about the color of skin in the beginning. They're after low-hanging fruit and maybe establishing facial recognition as 'proof enough'. A poor, white ex-con who can't afford a lawyer might just take a plea deal. I think the point of possible divergence is when the guy lawyer-ed up.
- truthwhisperer 6y ago
  Maybe most type of criminal activity in that category is done by black persons and they are harder to distinguish due to dark patterns ...
- Scoundreller 6y ago
  > it mostly false positives only for dark-skinned people. Dunno what the technical reason is, but if there was one, I bet it went something like this: v1: sales 'engineer' says: "Our FR product works great on light skin but terribly on darker skin so we've largely disabled that" and made zero sales. On v2 (version2 or vendor2), they just dialed up the acceptable false positive rate, said "ya, it works on everyone, no problem!" and sold sold sold!
  - pmiller2 6y ago
    Part of the reason is likely that black people, particularly dark skinned black people, are just harder to photograph than people of other ethnicities: https://www.npr.org/sections/codeswitch/2014/04/16/303721251/light-and-dark-the-racial-biases-that-remain-in-photography https://www.npr.org/sections/codeswitch/2014/04/16/303721251... Edit: I'm placing this higher up in the thread for more visibility, but, for those interested, here are some tips for photographing people with darker skin tones: https://creativecloud.adobe.com/discover/article/10-tips-for-photographing-darker-skin-tones https://creativecloud.adobe.com/discover/article/10-tips-for...
    - Scoundreller 6y ago
      That's fine, but the system shouldn't be dialed up to accept more false positive matches, but it sounds like it is being so.
      - pmiller2 6y ago
        Oh, I agree 100%. I'm just saying that maybe the quality of input to the algorithms leaves something to be desired. Or, maybe the fact that there are relatively few black people in tech means that the developers of these algorithms don't even notice that there's a bias to correct for, whether it's in the input or not.
    - dstick 6y ago
      Oh come on! “They” are harder to photograph? This is exactly the same like saying black people are less intelligent because they don’t hold the higher paid jobs. It’s not by choice. Nor is there any physical reason why that should be. The technology to photograph is biased to better capture white skinned people. It even says so in the very article you link to: “A lot of [the design of film and motion technology] was conceived with the idea of the best representation of white people.” Improve the technology!
- treeman79 6y ago
  If you want variety, iPhone had some trouble with Chinese. https://nypost.com/2017/12/21/chinese-users-claim-iphone-x-face-recognition-cant-tell-them-apart/ https://nypost.com/2017/12/21/chinese-users-claim-iphone-x-f...
  - pmiller2 6y ago
    Ugh. You would think Apple would do more testing in one of their largest markets, and the place where iPhones are literally built.
- throwaway2245 6y ago
  > I'm not quite ready to allow my cynicism to let me think it is an intentional feature Hanlon's razor is hinted at in the later discussion, i.e. "never attribute to malice that which is adequately explained by incompetence" (at least, both the words incompetence and malice appear under discussion) But, I prefer to apply a different rule-of-thumb: "incompetence is strongly equivalent to malice"
lb1lf 6y ago
Just out of curiosity - if the AI software relies on utilizing images scraped from the web, could one conceivably hit the developer with copyright infringement charges? Or would this be fair use?
- digdigdag 6y ago
  It's illegal. Prior authorization _should_ be required using someone's copyrighted work . In this case, the AI firm that compiled the profile of this poor soul probably had not reached out and asked for authorization for use of his image in a vast database which is then productized and sold to law enforcement agencies. Now, things get complicated when you involve firms which, in the process of signing up a user, might force an agreement to sell/market the data in a non-exclusive manner to "trusted partners". Sooooo...is it illegal? Depends on on the lawyer and how long you can fund a lawsuit for, I suppose.
  - ve55 6y ago
    We do not actually have precedence of this being illegal yet. You are not using the image in creation of a work (in which case, you need permission), but using it to train an AI model, which we currently have no legal indication is not allowed, even if the image is copyrighted.
    - YawningAngel 6y ago
      Could you explain to me why you don't consider Clearwater's product to be a work in this regard? It seems plausible to me that someone could infringe copyright by incorporating copyrighted works into a machine learning model
      - ve55 6y ago
        I don't have much of an opinion myself on it, I'm just trying to state that so far, this appears to be the status quo. Here is some recent additional information on it compiled by Gwern: https://www.gwern.net/Faces#copyright https://www.gwern.net/Faces#copyright (scroll up just above this link): >Models in general are generally considered “transformative works” and the copyright owners of whatever data the model was trained on have no copyright on the model. (The fact that the datasets or inputs are copyrighted is irrelevant, as training on them is universally considered fair use and transformative, similar to artists or search engines; see the further reading.) The model is copyrighted to whomever created it.
gengelbro 6y ago
How many people have been arrested historically with just a plain ol' case of mistaken identity (i.e. they just looked alike). Seems like we'd need to know the ambient rate to tell if we're really regressing.
- foobaw 6y ago
  I think the shock comes from the perception that facial recognition should have 0 false positives. Everyone would agree that it's better.
  - dxdm 6y ago
    No, the expectation of false positives should mean that a positive match is not enough to keep somebody locked up for multiple days.
    - fmajid 6y ago
      Specially if it’s someone who voluntarily turned himself in in the first place.
- cgriswald 6y ago
  This went on for a year before a judge finally told the prosecutor's office they need actual evidence beyond facial recognition and they subsequently dropped the charges.
kevin_thibedeau 6y ago
> Outside, the man jumped into a black Dodge Challenger as officers drew their firearms and told him to stop Not sure how this was justified for shoplifting.
- deleted 6y ago
  [deleted]
  - deleted 6y ago
    [deleted]
- speedgoose 6y ago
  Definitely not justified in my country.
- crooked-v 6y ago
  Police across the US are trained to default to the threat of violence at all times, with the expectation that if they don't someone will immediately try to kill them. A couple of examples: https://www.theatlantic.com/national/archive/2014/12/police-gun-shooting-training-ferguson/383681/ https://www.theatlantic.com/national/archive/2014/12/police-... https://www.startribune.com/fear-based-training-for-police-officers-is-challenged/487958041/ https://www.startribune.com/fear-based-training-for-police-o... https://www.minnpost.com/community-voices/2020/06/warrior-or-guardian-how-police-training-fails-us/ https://www.minnpost.com/community-voices/2020/06/warrior-or...
- WillPostForFood 6y ago
  When the police pulled out their handcuffs to arrest the man, he fled out the back door of the hotel, losing a sneaker on the way. Outside, the man jumped into a black Dodge Challenger as officers drew their firearms and told him to stop, but he took off, causing an officer to jump out of the way, the report states. He crashed into the back of a patrol car and a column outside the hotel before speeding out onto Route 9, an officer wrote. Not sure how you characterize that as "shoplifting". They had already questioned him about the shoplifting, without pulling their guns. It was the escape, driving at the officers, and ramming that got the guns pulled.
olefoo 6y ago
So if you are wondering why Clearview is so bad specifically for black people; it might help to know that the company has extensive ties to Right-wing white supremacists. And that many of those relatioinships were formed in Peter Thiel's orbit. https://artificialintelligence-news.com/2020/04/08/clearview-ai-found-extensive-ties-far-right/ https://artificialintelligence-news.com/2020/04/08/clearview...
- adrianscott 6y ago
  How would that help?
  - jhayward 6y ago
    Could you clarify your question? As I read it, it is essentially "I don't see how a close association with white supremacists/racists is relevant to the fact that it mis-identifies black people as criminals", which I doubt is what you mean.
- dghsjs 6y ago
  Even if clearview is 100% white supremacists they would still want the software to catch the correct brown man.
deleted 6y ago
[deleted]
option 6y ago
has any human reviewed the match before throwing the person in jail? I hope they did and in that case it is clearly a human error. Of course, no AI system (facial recognition or not) should be a sole decision maker for even detaining anyone.
TeeMassive 6y ago
They do look very alike, it's the kind of mistake a human would make. But that's not the problem. The fact that it is automated means these mistakes will happen a lot more, will target dark skinned people more and since a tool can't be fined or put to trial then nobody will be held accountable.
asymptotically4 6y ago
They should add this to the "In The News" section of clearview.ai's website.
- nelsonenzo 6y ago
  LOL. I love this idea.
StreamBright 6y ago
When is going to be a time when everybody understands that these algorithms are statistical and quite often wrong? It is always a surprise to me when I talk to somebody who does not understand neural nets or current state of ML algos and explain that it is just statistics and you cannot expect these systems to be 100% correct ever.
- crooked-v 6y ago
  The problem comes in the expectations set based on other industries, where correctness actually matters both ethically and in terms of legal liability. The corresponding XKCD comic: https://xkcd.com/2030/ https://xkcd.com/2030/
- _underfl0w_ 6y ago
  You might be surprised if you ever worked in B2B or B2C sales. It's not uncommon to apply a "spin" to effectively set unreasonable expectation for your client, whether they be an average joe or a law enforcement agency. Not ethical, but also not exactly uncommon.
- fmajid 6y ago
  And most people don’t understand Bayes’ rule and the impact of false positives.
99_00 6y ago
The problem is the police not the technology. Blaming the technology excuses these awful, incompetent police who will just screw up in a low tech way. Once the computer flagged a suspect a basic investigation should have ruled out or confirmed the suspect.
- grumio 6y ago
  Absolutely right. They need to have some skin in the game, or otherwise be held accountable if their actions cause harm. That can't be solved with a software update.
- ogre_codes 6y ago
  > The problem is the police not the technology. It's worth pointing out that the technology isn't capable of what people think it is capable of.
  - vrperson 6y ago
    I don't see much explaining of the capabilities of AI in the article.
    - ogre_codes 6y ago
      Pointing out a colossal failing of ML based facial recognition is the point of the article. Without that failure, there is no story here.
      - vrperson 6y ago
        Where exactly is the failure? It identified a person who looks similar to the person in the drivers license? It could even actually be the same person, given that the drivers license is assumed to be fake (so the fake could use the photograph of the suspect).
- MeinBlutIstBlau 6y ago
  That part boggled my mind. He had someone who could prove he was not who they were looking for. How the hell can the system just go ahead and charge you for something even though there is perfect evidence stating you are not who they are looking for!?
shadowprofile77 6y ago
Is there a non-paywalled version of this anywhere, or a way of finding it. Really, it's tiresome to see constant posting of content that most others can't even read.
- rendall 6y ago
  https://web.archive.org/web/20201228193320/https://www.nj.com/middlesex/2020/12/he-spent-10-days-in-jail-after-facial-recognition-software-led-to-the-arrest-of-the-wrong-man-lawsuit-says.html https://web.archive.org/web/20201228193320/https://www.nj.co...
- _underfl0w_ 6y ago
  It may be a jurisdictional thing or a lack of ad blocking? I didn't see a paywall here in the U.S. with a generic ad blocker on my phone, FWIW.
irateswami 6y ago
Perhaps it's just my schadenfreude acting up but I pray to sweet baby Jesus that they try to bust somebody with this. I'd love to see a whole bunch of engineers from Clearview AI get a proper shellacking on the stand from a criminal defense attorney. I don't see how anything from this product could be used as evidence without the algorithms being questioned or exposed in court.
- sacred_numbers 6y ago
  That's not what would happen, though. The police will use this tech on poor people who can't afford their own attorneys, and public defenders are too overworked, underpaid, and/or cynical to try to push back against the system. On the off chance that a rich person gets falsely arrested due to this tech the prosecution will probably immediately settle out of court to avoid scrutiny. This same scenario has played out over the last decade or so with the Stingray device[1] in particular and various forms of warrantless (or essentially warrantless) wiretapping in general. Even if facial recognition tech is officially banned or restricted, it will probably be used secretly and covered up using parallel construction[2]. [1]https://en.wikipedia.org/wiki/Stingray_phone_tracker https://en.wikipedia.org/wiki/Stingray_phone_tracker [2]https://en.wikipedia.org/wiki/Parallel_construction https://en.wikipedia.org/wiki/Parallel_construction
- LanceH 6y ago
  How many people have these engineers been arresting? This is news to me. I'm shocked!! They don't have the power to arrest people. How is this being allowed?!
- wyldfire 6y ago
  The facial recognition is likely 'only' used as a pursuit/detection mechanism. Presumably/hopefully when making a case in court they use traditional evidence like eyewitness identification, DNA, etc. Novel evidence like this is probably unlikely to do well in court for the reasons you describe.
  - greggyb 6y ago
    In the specific instance described in the article, the police and prosecutor failed to produce any more evidence than the facial recognition match. Nevertheless, they imprisoned a man for ten days, and proceeded to press for criminal charges over the course of the year. As described the man had to spend his entire savings on this defense. The court system worked correctly, in that the judge told the prosecutor to produce more evidence. The charges were dismissed. So we have the "happy" ending of a man whose life has been turned upside down, whose financial situation has been ruined, and who has had to spend his time and effort defending himself rather than contributing to society in a positive way. All because facial recognition was used as the only piece of evidence.
_underfl0w_ 6y ago
Does Clearview AI train on mugshot datasets specifically? I'm absolutely NOT advocating for its use at all, but it seems like that might be a good source for well-tagged, organized data for what is turning out to be a very problematic (though apparently allowed) use case. From a photography standpoint, it seems to follow that identical lighting in the same room where they're taking everyone's mugshots could produce different levels of contrast for facial features across different skin tones, simply as an artifact of cheap lenses and a lack of white-balancing or overall care, really. If persons A and B have different skin tones, then the routine, careless, terrible, one-size-fits-all black-and-white photo by an underpaid government office worker may not accurately capture the contrast in the shadows of both of their facial features to the same degree, no futile human bias required. This lack of definition may then further help promote any existing biases in ML training, enforcement, etc. towards people whose features aren't as well contrasted in the resultant, awful photos. Perhaps training on such a grainy, washed-out dataset would at least help the ML distinguish smaller variances in contrast to a finer degree, if nothing else. Us humans can do it, after all.
- Cactus2018 6y ago
  > Does Clearview AI train on mugshot datasets specifically? https://www.theverge.com/2020/2/6/21126063/facebook-clearview-ai-image-scraping-facial-recognition-database-terms-of-service-twitter-youtube https://www.theverge.com/2020/2/6/21126063/facebook-clearvie... * "Facebook and LinkedIn are latest to demand Clearview stop scraping images for facial recognition tech. Twitter and YouTube have also objected."
eecc 6y ago
Perhaps USians as desensitized to guns but I took pause when reading that the police officers drew their guns when chasing the fugitive in the original incident, about shoplifting candies from a hotel lounge? (And I wonder if the concierge that called the cops on this guy hadn’t provoked a taunt with some “kind words”.)
- renewiltord 6y ago
  Americans are, as a rule, highly authoritarian. You'll see it online as people use pseudo-bureaucratic corpo-speak to justify state violence: "refusal to obey a lawful order", "did not comply with a lawful request", "in violation of state regulation" etc. Having lived elsewhere and in the US (a country I practically adore), I suspect it is because Americans have not really had true authoritarianism in any sense at any point in their history. Every time they stray close to the darkness and walk away unscathed and it convinces them that they can never really be consumed by it so they're willing to walk its very edge. Perhaps the UK having a monarch and a state religion is a visible reminder of what a monstrosity the state can be. It's funny that both nations have populations that support the massive expansion of the state into human affairs: it's just that the Americans want it to enforce behaviour and the British want it to redistribute wealth. The most amusing manifestation of the American comfort with violence is the difference in how streakers are handled in British football games and American football games. The former have overweight stewards lumber after naked people, failing repeatedly to apprehend, eventually accompanying them off the field. The latter have large, muscular guards catch up to and violently tackle the streaker to the ground.
  - watwut 6y ago
    > Americans have not really had true authoritarianism in any sense at any point in their histor That is not really true. American history is constant struggle between authoritarianism and opposition.
banjomet 6y ago
"The software, which was created by Clearview Al, was criticized for its heavy reliance on billions of social media photos to identify criminal suspects." So how exactly did they get access to all photos? It must have been trained on public networks like instagram, not facebook, right?!? If it was FB that would be interesting.
- Cactus2018 6y ago
  https://www.theverge.com/2020/2/6/21126063/facebook-clearview-ai-image-scraping-facial-recognition-database-terms-of-service-twitter-youtube https://www.theverge.com/2020/2/6/21126063/facebook-clearvie... > "Facebook and LinkedIn are latest to demand Clearview stop scraping images for facial recognition tech. Twitter and YouTube have also objected." https://slate.com/technology/2020/02/youtube-linkedin-and-others-serve-clearview-ai-with-cease-and-desist-letters.html https://slate.com/technology/2020/02/youtube-linkedin-and-ot... > "Facebook notably has not sent a formal cease-and-desist letter but claims to have sent other letters to Clearview to request more detail on its practices and then eventually “demanded” that it stop scraping user data. Peter Thiel, a venture capitalist and notable surveillance enthusiast who sits on Facebook’s board of directors, invested $200,000 in Clearview’s first round of funding."
upofadown 6y ago
>...notified Woodbridge police they had a “high profile” match to the photo,... That's weird. You would think a facial recognition system would return hundreds of matches in a case where it was searching through a whole lot of images. What does "high profile" mean in this case?
drak0n1c 6y ago
In Shenzhen, starting in the last couple years if you jaywalk your face is automatically recognized and the fine is automatically deducted from your bank account. It's extremely accurate due to full access to corroborating location/transit/payment data. This happens within 20 seconds, and even includes expats. Your face was permanently indexed during the passport check. Such an automated system at first glance may seem like it reduces on-the-ground human flaws and bias, but it actually enables far deeper corruption. Those with administrative control would be able to selectively highlight or concoct any sort of offense to undermine their personal or political opponents. "Show me the man and I'll show you the crime" - Lavrentiy Beria, Stalin's Chief of Secret Police
- daniellarusso 6y ago
  Does this work with sunglasses and facemasks?
  - pmiller2 6y ago
    Facial recognition can work with ordinary facemasks. I don't have a link handy, but I'm sure Google can produce one, given the timeliness of the question. As to your question, I also wonder about such things as: * Makeup: https://www.vogue.com/article/anti-surveillance-makeup-cv-dazzle-protest https://www.vogue.com/article/anti-surveillance-makeup-cv-da... * Hair styles: https://cvdazzle.com/ https://cvdazzle.com/ * Clothes: https://www.businessinsider.com/clothes-accessories-that-outsmart-facial-recognition-tech-2019-10#belgian-scientists-developed-a-prototype-for-a-graphic-print-that-could-be-added-to-clothing-to-attack-and-baffle-surveillance-technology-11 https://www.businessinsider.com/clothes-accessories-that-out... * Infrared LEDs: https://www.schneier.com/blog/archives/2018/03/fooling_face_re.html https://www.schneier.com/blog/archives/2018/03/fooling_face_... * Image manipulation software: https://www.theverge.com/2020/8/4/21353810/facial-recognition-block-ai-selfie-cloaking-fawkes https://www.theverge.com/2020/8/4/21353810/facial-recognitio... And other things.
    - Retric 6y ago
      The first several links all say they make a big difference. Face masks are already known to stop the spread of coronavirus. Apparently, they can also make it much harder for facial-recognition software to identify you, too. (cnn) It’s true accurate identification in ideal conditions is still possible, but catching random jaywalkers is hardly ideal conditions.
11thEarlOfMar 6y ago
It was about 10 days after the incident that he was in custody. What did the 3 eye witnesses say? Did they make positive identifications? Did the officers involved in the initial incident claim he was the same person? Were any of the 7 officers the same ones who encountered him in the hotel? How about the hotel staffer? Did they make a positive ID? Was 'Jamal' a hotel guest? Seems like there should be a discussion about all of this in the article. Facial recognition as the sole evidence must not be used to make arrests.
- wvenable 6y ago
  Eye witness identifications are very likely worse than facial recognition. There should be some other evidence than either of those.
agnosticmantis 6y ago
Are the creators of these algorithms penalized for their false positives? I think the confidence levels returned by these algorithms would be quickly calibrated if there was some strict penalty for high-confidence false positives.
- KittenInABox 6y ago
  I would prefer that there is a strict penalty for people who rely too much on their tool that they can't even do their jobs correctly. I'd fire a carpenter if an AI told the carpenter to fuck up my cabinet, because I expect my carpenter to double-check. Similarly if the carpenter finds the AI consistently useless and an avenue to get their ass sued, then they'll decline to purchase shit AI.
  - agnosticmantis 6y ago
    I agree. The penalty on the user of AI would in turn incentivize the user to penalize/reward the AI developer. But I’m not sure if it’s reasonable to require every user to be able to assess the long run accuracy of a statistical system given their limited interaction. Maybe there’s a market for companies to reliably assess and certify AI systems or by creating a sort of a prediction market that penalize overconfident false positives.
  - deleted 6y ago
    [deleted]
- deleted 6y ago
  [deleted]
- LanceH 6y ago
  Why should they be? This system is saying, "of the 10 million people we have in our database, this one looks like the photo you gave us." It is the police and prosecutor who are asserting, "this person did it and should be held in jail." This tool is a visual equivalent of looking up someone's name and is the visual equivalent of the no-fly-list being used as though no two people share the same name.
okthenletssee 6y ago
Is it really so uncommon for suspects of a crime to spend time in jail? I think I have read such stories without computers having anything to do with it. It's just a sensationalist spin trying to ride the AI scare.
diebeforei485 6y ago
What judge signed the arrest warrant, and why are they still a judge?
DubiousPusher 6y ago
It seems to me this is not a facial recognition story but a story about how the state, with very little evidence, can ruin your life and leave you with very little recourse. As concerned as I am about state use of facial recognition, these kind of things have been happening to people for decades now. The solution to this problem is to better secure the rights of people accused of crimes. We need to ensure that people accused do not lose their jobs and homes. That they have fair opportunity to communicate with counsel and family (both are often needlessly limited by DoC). And the state must be responsible to mitigate its mistakes. Bail requirements are too often granted as par for the course and need a "burden of proof" type rethinking. Holding people at home should be required before incarceration. Lastly we need to remove the power of the legislatures to unequally empower prosecutors and public defense attorneys.
- js2 6y ago
  > a story about how the state, with very little evidence, can ruin your life and leave you with very little recourse. Buttle? Tuttle? https://vimeo.com/224785800 https://vimeo.com/224785800
- Blikkentrekker 6y ago
  > It seems to me this is not a facial recognition story but a story about how the state, with very little evidence, can ruin your life and leave you with very little recourse. The facial recognition is relevant, as they relied on facial recognition software, though it is apparently illegal in New Jersey. From the article as well: > He asked for a lawyer, then was taken to a hallway where he was handcuffed to a bench. About an hour later, the officers – he counted seven – told him they were going to take him to a different room for more questions. As far as I understand U.S.A. law, when a suspect ask for a lawyer, the interrogation is to stop immediately, and a lawyer must be provided or he must be allowed to call one, and the lawyer must be præsent ere the interrogation continue. The real takeaway seems to be that the police did something illegal. The rest of your post is about many more rules, but the issue is that the rules were broken here.
  - tinalumfoil 6y ago
renewiltord 6y ago
The problem appears to be that police and prosecutors are either morons or incentivized so strongly to arrest and convict that they ignore obvious nonsense. Given that this is the case, the use of AI cannot be accepted by any members of the US justice system. If I had access to a facial recognition system, I would use it as a whittling down tool, to reduce the amount of hay to find the needle in. But they clearly see a straw and conclude that it is a needle. It's like giving a four year old a handgun. It's just not responsible. They're just not as good at this as I am. So no handgun for you. It's a pity. Ideally you should be able to discriminate, and intelligent police should be given the tool, and stupid police not. However, because stupid police and smart police exist in an emulsion, it is better to just waste the time of the smart police than to give the stupid police too powerful a tool.
- MeinBlutIstBlau 6y ago
  When the state doesn't have to foot the bill taking you to court, of course they don't care. If the procsecutor had to pay all your legal fees if they lost, we'd either get some pretty amazing state prosecutors or ones who who only accept evidence from cops that can convince a judge. It's almost as if you stop cops from just locking people up it creates an immense snowball effect on the whole system...
- tolbish 6y ago
  > If I had access to a facial recognition system, I would use it as a whittling down tool, to reduce the amount of hay to find the needle in. But they clearly see a straw and conclude that it is a needle. It's like giving a four year old a handgun. It's just not responsible. They're just not as good at this as I am. So no handgun for you. Just wanted to say I've never heard such an argument worded so well. What is the course of action for the common man other than verbalizing such thoughts to local city councils and law enforcement?
  - renewiltord 6y ago
    I think the local powers have figured out the route that will already succeed: blame the tool. That way permits everyone to leave with their egos intact. The Art of War 7.36 in action, one might say.
flatus 6y ago
That's nothing: Steven Talley was identified by the FBI as the primary suspect in two bank robberies using a facial recognition algorithm. He had an iron-clad alibi, but the police and FBI weren't convinced. In court one of the bank tellers said Talley definitely wasn't the robber. Nonetheless Talley lost his job, his wife and his family and was held in prison for months: "LOSING FACE: How a Facial Recognition Mismatch Can Ruin Your Life" https://theintercept.com/2016/10/13/how-a-facial-recognition-mismatch-can-ruin-your-life/ https://theintercept.com/2016/10/13/how-a-facial-recognition... FTFA: "Talley said he was held for nearly two months in a maximum security pod and was released only after his public defender obtained his employer’s surveillance records. In a time-stamped audio recording from 11:12 a.m. on the day of the May robbery, Talley could be heard at his desk trying to sell mutual funds to a potential client." Today Talley is still trying to claw his way back to normalcy. In the outstanding book "Hello World" Hannah Fry examines Talley's story as part of a chapter on crime, AI and facial recognition. Reading Fry's book convinced me that facial recognition software simply does not work well enough to use in police work. As Fry says: "If you're searching for a particular criminal in digital line-up of millions...the best-case scenario is that you won't find the right person one in six times...". That is not nearly good enough for law enforcement and the courts. "Hello World" by Hannah Fry https://www.amazon.com/Hello-World-Hannah-Fry/dp/0857525255 https://www.amazon.com/Hello-World-Hannah-Fry/dp/0857525255
- vrperson 6y ago
  Again, how is that story a failure of AI/face recognition? At some point they even have a human compare the likeness, and the human also concludes it is the same person. The article even features the sentence "teve Talley is hardly the first person to be arrested for the errors of a forensic evaluation." And yet people seem to be hellbent to make it about AI.
  - mahkeiro 6y ago
    It’s a failure in the sense that people put too much confidence in this kind of algorithm and put them above any eye witness when they should just be considered as another element.
duxup 6y ago
What happens if this went to trial (let's say there was enough other evidence)? How do you question your accuser? Do you have access to the software? How much access? Do many people even have the resources to examine the software?
pacamara619 6y ago
"Software problem, nothing we could do." Every time someone fucks up using technology and noone is held accountable, that's what we hear.

259 points by sharkweek 6y ago

I wish I could say that I was shocked that the victim of bad facial recognition is black. I wish I could say that I'm shocked that authorities take the word of software over looking at the evidence and thinking, "umm, doesn't look like him..." Or perhaps someone did look and said, "meh, they all look the same to me." All I know is that I'm highly skeptical that this white guy would suffer the same fate, even if I were to have a similar criminal record. Facial recognition seems to be the polygraph for a new century. But unlike a polygraph, it mostly false positives only for dark-skinned people. While I'm not quite ready to allow my cynicism to let me think it is an intentional feature, I question how many are demanding a fix.

AmVess 6y ago
The last thing police need are more tools that don't work and they don't understand how to use. That they still use polygraphs should make everyone's hair stand on end.
- tartoran 6y ago
  As I understand polygraphs are used like placebos/intimidation tactic to get the suspects to spill the beans. As far as I know the polygraph isn't accepted as evidence in court unless all parties agree to it. However, the dirtiest thing is to use all kind useless evidence and intimidation techniques to scare otherwise innocent people into taking plea deals. This whole guilty plea saga is a disgrace to the justice system.
  - jonny_eh 6y ago
    > As far as I know the polygraph isn't accepted as evidence in court unless all parties agree to it That's not the only place where they can be used, and cause damage.
  - _underfl0w_ 6y ago
    Correct. It's widely considered inadmissible as evidence in court. They're also rarely used outside of the U.S. AFAIK. Recordings of interrogations that feature them also show that they tend to come along with other appeals to ethos ("this is doctor so-and-so...") and hand-wavey forms of intimidation like touting their "seriousness" or "efficacy". Anything that requires a person to "interpret" its output is likely not very deterministic, and so likely not very reliable for establishing... anything. This is akin to palm reading or graphology. Unfortunately, revealing to the police in such a situation exactly how foolish they look by getting behind such a device isn't exactly advisable either!
    - amiga 6y ago
      The Penn and Teller series, Bullshit, features an episode on polygraph machines and interviewer techniques. It's a good watch.
  - KittenInABox 6y ago
    A failed polygraph while inadmissible in court will still be used by police during their own investigations to confirm or deny their "gut feeling".
    - AmVess 6y ago
      I failed a polygraph. There was a crime at a store, and my name came up when they searched through the ATM records. Police had ample evidence that I was in no way related to the crime, but kept trying to find something to pin on me because of the failed test. Those particular detectives had complete faith in the lie detector as a valid tool of police work. Video evidence of me entering the store, using the atm, and immediately leaving the store? That was not evidence to them because I failed the polygraph. I take an exceedingly dim view of any police department that uses a polygraph in any capacity. They've been proven time and time and time again to be worthless as a tool, but police still use them. The rough equivalent is using tarot cards as a tool for solving crimes.
      - CraneWorm 6y ago
        You assume they wanted truth when in reality they only want confession.
      - syshum 6y ago
        There are many things in law enforcement that have either been proven to be worthless, near worthless or never been proven to be valid... Eye Witness Testimony for example, aside from the fact the people are terrible at recall anyway and High Stress causes memory problems above and beyond the normal memory issues of the human mind, it is very very very easy to manipulate a victim either directly or indirectly to identify the "target" no necessarily the actual criminal police "drug" dogs are another one, it is more or less a blank check to probable cause for a search. Then there is one people never even question because it is the foundation of the legal system, the good old finger print... It always amazes me that with DNA it is always probabilistic, "1 in 4 billion people will have this same DNA" but with Finger Prints is an absolute, Match no match.. Call me crazy but I don't think that position is scientifically valid
  - throwaway201103 6y ago
    > the dirtiest thing is to use all kind useless evidence and intimidation techniques to scare otherwise innocent people into taking plea deals They don't even need to do that. They can just lie to you. "Hey, your buddy just totally ratted you out in the other room, you better tell us your side of it if you don't want to take the rap for everything"
    - SerLava 6y ago
      Even worse, they can pretend they have evidence that they don't, pretend they believe the evidence, and pretend it's enough to convict you by itself. They can tell you to just say you were there, and they'll go easy on you. You lie and say you were there, now they have evidence that can actually lock you up, even if you are a random person off the street.
      - bluejekyll 6y ago
        To a degree, but I also think this will be changing over time as more and more innocent people are found to have been convicted on false statements used through coercive lying tactics by the police. For example, in this case police threatened to arrest the mans wife if he didn’t confess to killing his child (a new trial was ordered): https://brooklyneagle.com/articles/2014/02/20/police-can-lie-to-suspects-during-interrogations-says-court-of-appeals/ https://brooklyneagle.com/articles/2014/02/20/police-can-lie... But they went on to say it was still acceptable to lie to suspects... I imagine this murky area of the law will eventually say that police cannot lie to suspects. The Central Park Five is mentioned in that article and is a blatant example of police lying to coerce testimony and then putting innocent children in prison.
      - syshum 6y ago
        Even Worse than all of that, they can use all of those things, while threatening you with dozens of charges and decades of imprisonment then say "but if you act right now and sign this we will only give you a couple of years and then you will be free" in any other context these tactics themselves would be illegal, and i am firm believer that you can not enforce the law by breaking, and that the police should NEVER how the power to do things the public can not
- pmiller2 6y ago
  See also "touch DNA": https://www.techdirt.com/articles/20191026/12493943266/federal-court-says-touch-dna-analysis-is-mostly-guesswork-that-cant-be-used-as-evidence.shtml https://www.techdirt.com/articles/20191026/12493943266/feder...
gwern 6y ago
Are we looking at the same photos? They look uncannily alike to me, to the point where I'm left wondering if the forged license actually reuses a photo from Parks. (As a double convict in NJ, I suspect his earlier mugshots would not be hard for the forger to get. Certainly no harder than forging a license.)
- crooked-v 6y ago
  "Uncannily"? I don't see it. Different mouth proportions compared to the nose, different cheekbones, different eyebrow lengths...
- sebastien_b 6y ago
  They do look similar (enough to become a potential suspect, though I don't think should be enough for a warrant), and given that police departments don't "protect" their mugshots, I wouldn't be surprised if the picture was indeed reused (or photoshopped). But I really wonder if the same officers that dealt with the suspect also questioned the innocent victim in this case - it seems like this wouldn't be the case (and gross negligence on their part) - those people should've been able to tell them apart (instead of solely relying on the picture they got from the license). I think this is what the judge wanted instead of just the facial recognition "evidence" (which eventually led to the charges being dropped) - something like, oh I don't know, the fingerprints from the abandoned vehicle? Basic investigative work, perhaps?
  - pmiller2 6y ago
    Eh, I wouldn't even say that. I'd say they look similar enough to be put into the same photo lineup, but I wouldn't even drag the guy from the article in to do a physical lineup, much less arrest him.
    - sebastien_b 6y ago
      Agree that a photo alone shouldn’t be enough for an arrest warrant. Like I said - either the officers that dealt with the suspect didn’t deal with/look at the victim, or else they’re blind (or trying to meet a quota).
cgriswald 6y ago
> All I know is that I'm highly skeptical that this white guy would suffer the same fate, even if I were to have a similar criminal record. It is not clear to me that a prosecutor's office which has demonstrated that it is not interested in justice, would have cared about the color of skin in the beginning. They're after low-hanging fruit and maybe establishing facial recognition as 'proof enough'. A poor, white ex-con who can't afford a lawyer might just take a plea deal. I think the point of possible divergence is when the guy lawyer-ed up.
truthwhisperer 6y ago
Maybe most type of criminal activity in that category is done by black persons and they are harder to distinguish due to dark patterns ...
Scoundreller 6y ago
> it mostly false positives only for dark-skinned people. Dunno what the technical reason is, but if there was one, I bet it went something like this: v1: sales 'engineer' says: "Our FR product works great on light skin but terribly on darker skin so we've largely disabled that" and made zero sales. On v2 (version2 or vendor2), they just dialed up the acceptable false positive rate, said "ya, it works on everyone, no problem!" and sold sold sold!
- pmiller2 6y ago
  Part of the reason is likely that black people, particularly dark skinned black people, are just harder to photograph than people of other ethnicities: https://www.npr.org/sections/codeswitch/2014/04/16/303721251/light-and-dark-the-racial-biases-that-remain-in-photography https://www.npr.org/sections/codeswitch/2014/04/16/303721251... Edit: I'm placing this higher up in the thread for more visibility, but, for those interested, here are some tips for photographing people with darker skin tones: https://creativecloud.adobe.com/discover/article/10-tips-for-photographing-darker-skin-tones https://creativecloud.adobe.com/discover/article/10-tips-for...
  - Scoundreller 6y ago
    That's fine, but the system shouldn't be dialed up to accept more false positive matches, but it sounds like it is being so.
    - pmiller2 6y ago
      Oh, I agree 100%. I'm just saying that maybe the quality of input to the algorithms leaves something to be desired. Or, maybe the fact that there are relatively few black people in tech means that the developers of these algorithms don't even notice that there's a bias to correct for, whether it's in the input or not.
  - dstick 6y ago
    Oh come on! “They” are harder to photograph? This is exactly the same like saying black people are less intelligent because they don’t hold the higher paid jobs. It’s not by choice. Nor is there any physical reason why that should be. The technology to photograph is biased to better capture white skinned people. It even says so in the very article you link to: “A lot of [the design of film and motion technology] was conceived with the idea of the best representation of white people.” Improve the technology!
treeman79 6y ago
If you want variety, iPhone had some trouble with Chinese. https://nypost.com/2017/12/21/chinese-users-claim-iphone-x-face-recognition-cant-tell-them-apart/ https://nypost.com/2017/12/21/chinese-users-claim-iphone-x-f...
- pmiller2 6y ago
  Ugh. You would think Apple would do more testing in one of their largest markets, and the place where iPhones are literally built.
throwaway2245 6y ago
> I'm not quite ready to allow my cynicism to let me think it is an intentional feature Hanlon's razor is hinted at in the later discussion, i.e. "never attribute to malice that which is adequately explained by incompetence" (at least, both the words incompetence and malice appear under discussion) But, I prefer to apply a different rule-of-thumb: "incompetence is strongly equivalent to malice"

lb1lf 6y ago

Just out of curiosity - if the AI software relies on utilizing images scraped from the web, could one conceivably hit the developer with copyright infringement charges? Or would this be fair use?

digdigdag 6y ago
It's illegal. Prior authorization _should_ be required using someone's copyrighted work . In this case, the AI firm that compiled the profile of this poor soul probably had not reached out and asked for authorization for use of his image in a vast database which is then productized and sold to law enforcement agencies. Now, things get complicated when you involve firms which, in the process of signing up a user, might force an agreement to sell/market the data in a non-exclusive manner to "trusted partners". Sooooo...is it illegal? Depends on on the lawyer and how long you can fund a lawsuit for, I suppose.
- ve55 6y ago
  We do not actually have precedence of this being illegal yet. You are not using the image in creation of a work (in which case, you need permission), but using it to train an AI model, which we currently have no legal indication is not allowed, even if the image is copyrighted.
  - YawningAngel 6y ago
    Could you explain to me why you don't consider Clearwater's product to be a work in this regard? It seems plausible to me that someone could infringe copyright by incorporating copyrighted works into a machine learning model
    - ve55 6y ago
      I don't have much of an opinion myself on it, I'm just trying to state that so far, this appears to be the status quo. Here is some recent additional information on it compiled by Gwern: https://www.gwern.net/Faces#copyright https://www.gwern.net/Faces#copyright (scroll up just above this link): >Models in general are generally considered “transformative works” and the copyright owners of whatever data the model was trained on have no copyright on the model. (The fact that the datasets or inputs are copyrighted is irrelevant, as training on them is universally considered fair use and transformative, similar to artists or search engines; see the further reading.) The model is copyrighted to whomever created it.

gengelbro 6y ago

How many people have been arrested historically with just a plain ol' case of mistaken identity (i.e. they just looked alike). Seems like we'd need to know the ambient rate to tell if we're really regressing.

foobaw 6y ago
I think the shock comes from the perception that facial recognition should have 0 false positives. Everyone would agree that it's better.
- dxdm 6y ago
  No, the expectation of false positives should mean that a positive match is not enough to keep somebody locked up for multiple days.
  - fmajid 6y ago
    Specially if it’s someone who voluntarily turned himself in in the first place.
cgriswald 6y ago
This went on for a year before a judge finally told the prosecutor's office they need actual evidence beyond facial recognition and they subsequently dropped the charges.

kevin_thibedeau 6y ago

> Outside, the man jumped into a black Dodge Challenger as officers drew their firearms and told him to stop Not sure how this was justified for shoplifting.

deleted 6y ago
[deleted]
- deleted 6y ago
  [deleted]
speedgoose 6y ago
Definitely not justified in my country.
crooked-v 6y ago
Police across the US are trained to default to the threat of violence at all times, with the expectation that if they don't someone will immediately try to kill them. A couple of examples: https://www.theatlantic.com/national/archive/2014/12/police-gun-shooting-training-ferguson/383681/ https://www.theatlantic.com/national/archive/2014/12/police-... https://www.startribune.com/fear-based-training-for-police-officers-is-challenged/487958041/ https://www.startribune.com/fear-based-training-for-police-o... https://www.minnpost.com/community-voices/2020/06/warrior-or-guardian-how-police-training-fails-us/ https://www.minnpost.com/community-voices/2020/06/warrior-or...
WillPostForFood 6y ago
When the police pulled out their handcuffs to arrest the man, he fled out the back door of the hotel, losing a sneaker on the way. Outside, the man jumped into a black Dodge Challenger as officers drew their firearms and told him to stop, but he took off, causing an officer to jump out of the way, the report states. He crashed into the back of a patrol car and a column outside the hotel before speeding out onto Route 9, an officer wrote. Not sure how you characterize that as "shoplifting". They had already questioned him about the shoplifting, without pulling their guns. It was the escape, driving at the officers, and ramming that got the guns pulled.

olefoo 6y ago

So if you are wondering why Clearview is so bad specifically for black people; it might help to know that the company has extensive ties to Right-wing white supremacists. And that many of those relatioinships were formed in Peter Thiel's orbit. https://artificialintelligence-news.com/2020/04/08/clearview-ai-found-extensive-ties-far-right/ https://artificialintelligence-news.com/2020/04/08/clearview...

adrianscott 6y ago
How would that help?
- jhayward 6y ago
  Could you clarify your question? As I read it, it is essentially "I don't see how a close association with white supremacists/racists is relevant to the fact that it mis-identifies black people as criminals", which I doubt is what you mean.
dghsjs 6y ago
Even if clearview is 100% white supremacists they would still want the software to catch the correct brown man.

deleted 6y ago

[deleted]

option 6y ago

has any human reviewed the match before throwing the person in jail? I hope they did and in that case it is clearly a human error. Of course, no AI system (facial recognition or not) should be a sole decision maker for even detaining anyone.

TeeMassive 6y ago

They do look very alike, it's the kind of mistake a human would make. But that's not the problem. The fact that it is automated means these mistakes will happen a lot more, will target dark skinned people more and since a tool can't be fined or put to trial then nobody will be held accountable.

asymptotically4 6y ago

They should add this to the "In The News" section of clearview.ai's website.

nelsonenzo 6y ago
LOL. I love this idea.

StreamBright 6y ago

When is going to be a time when everybody understands that these algorithms are statistical and quite often wrong? It is always a surprise to me when I talk to somebody who does not understand neural nets or current state of ML algos and explain that it is just statistics and you cannot expect these systems to be 100% correct ever.

crooked-v 6y ago
The problem comes in the expectations set based on other industries, where correctness actually matters both ethically and in terms of legal liability. The corresponding XKCD comic: https://xkcd.com/2030/ https://xkcd.com/2030/
_underfl0w_ 6y ago
You might be surprised if you ever worked in B2B or B2C sales. It's not uncommon to apply a "spin" to effectively set unreasonable expectation for your client, whether they be an average joe or a law enforcement agency. Not ethical, but also not exactly uncommon.
fmajid 6y ago
And most people don’t understand Bayes’ rule and the impact of false positives.

99_00 6y ago

The problem is the police not the technology. Blaming the technology excuses these awful, incompetent police who will just screw up in a low tech way. Once the computer flagged a suspect a basic investigation should have ruled out or confirmed the suspect.

grumio 6y ago
Absolutely right. They need to have some skin in the game, or otherwise be held accountable if their actions cause harm. That can't be solved with a software update.
ogre_codes 6y ago
> The problem is the police not the technology. It's worth pointing out that the technology isn't capable of what people think it is capable of.
- vrperson 6y ago
  I don't see much explaining of the capabilities of AI in the article.
  - ogre_codes 6y ago
    Pointing out a colossal failing of ML based facial recognition is the point of the article. Without that failure, there is no story here.
    - vrperson 6y ago
      Where exactly is the failure? It identified a person who looks similar to the person in the drivers license? It could even actually be the same person, given that the drivers license is assumed to be fake (so the fake could use the photograph of the suspect).
MeinBlutIstBlau 6y ago
That part boggled my mind. He had someone who could prove he was not who they were looking for. How the hell can the system just go ahead and charge you for something even though there is perfect evidence stating you are not who they are looking for!?

shadowprofile77 6y ago

Is there a non-paywalled version of this anywhere, or a way of finding it. Really, it's tiresome to see constant posting of content that most others can't even read.

rendall 6y ago
https://web.archive.org/web/20201228193320/https://www.nj.com/middlesex/2020/12/he-spent-10-days-in-jail-after-facial-recognition-software-led-to-the-arrest-of-the-wrong-man-lawsuit-says.html https://web.archive.org/web/20201228193320/https://www.nj.co...
_underfl0w_ 6y ago
It may be a jurisdictional thing or a lack of ad blocking? I didn't see a paywall here in the U.S. with a generic ad blocker on my phone, FWIW.

irateswami 6y ago

Perhaps it's just my schadenfreude acting up but I pray to sweet baby Jesus that they try to bust somebody with this. I'd love to see a whole bunch of engineers from Clearview AI get a proper shellacking on the stand from a criminal defense attorney. I don't see how anything from this product could be used as evidence without the algorithms being questioned or exposed in court.

sacred_numbers 6y ago
That's not what would happen, though. The police will use this tech on poor people who can't afford their own attorneys, and public defenders are too overworked, underpaid, and/or cynical to try to push back against the system. On the off chance that a rich person gets falsely arrested due to this tech the prosecution will probably immediately settle out of court to avoid scrutiny. This same scenario has played out over the last decade or so with the Stingray device[1] in particular and various forms of warrantless (or essentially warrantless) wiretapping in general. Even if facial recognition tech is officially banned or restricted, it will probably be used secretly and covered up using parallel construction[2]. [1]https://en.wikipedia.org/wiki/Stingray_phone_tracker https://en.wikipedia.org/wiki/Stingray_phone_tracker [2]https://en.wikipedia.org/wiki/Parallel_construction https://en.wikipedia.org/wiki/Parallel_construction
LanceH 6y ago
How many people have these engineers been arresting? This is news to me. I'm shocked!! They don't have the power to arrest people. How is this being allowed?!
wyldfire 6y ago
The facial recognition is likely 'only' used as a pursuit/detection mechanism. Presumably/hopefully when making a case in court they use traditional evidence like eyewitness identification, DNA, etc. Novel evidence like this is probably unlikely to do well in court for the reasons you describe.
- greggyb 6y ago
  In the specific instance described in the article, the police and prosecutor failed to produce any more evidence than the facial recognition match. Nevertheless, they imprisoned a man for ten days, and proceeded to press for criminal charges over the course of the year. As described the man had to spend his entire savings on this defense. The court system worked correctly, in that the judge told the prosecutor to produce more evidence. The charges were dismissed. So we have the "happy" ending of a man whose life has been turned upside down, whose financial situation has been ruined, and who has had to spend his time and effort defending himself rather than contributing to society in a positive way. All because facial recognition was used as the only piece of evidence.

_underfl0w_ 6y ago

Does Clearview AI train on mugshot datasets specifically? I'm absolutely NOT advocating for its use at all, but it seems like that might be a good source for well-tagged, organized data for what is turning out to be a very problematic (though apparently allowed) use case. From a photography standpoint, it seems to follow that identical lighting in the same room where they're taking everyone's mugshots could produce different levels of contrast for facial features across different skin tones, simply as an artifact of cheap lenses and a lack of white-balancing or overall care, really. If persons A and B have different skin tones, then the routine, careless, terrible, one-size-fits-all black-and-white photo by an underpaid government office worker may not accurately capture the contrast in the shadows of both of their facial features to the same degree, no futile human bias required. This lack of definition may then further help promote any existing biases in ML training, enforcement, etc. towards people whose features aren't as well contrasted in the resultant, awful photos. Perhaps training on such a grainy, washed-out dataset would at least help the ML distinguish smaller variances in contrast to a finer degree, if nothing else. Us humans can do it, after all.

Cactus2018 6y ago
> Does Clearview AI train on mugshot datasets specifically? https://www.theverge.com/2020/2/6/21126063/facebook-clearview-ai-image-scraping-facial-recognition-database-terms-of-service-twitter-youtube https://www.theverge.com/2020/2/6/21126063/facebook-clearvie... * "Facebook and LinkedIn are latest to demand Clearview stop scraping images for facial recognition tech. Twitter and YouTube have also objected."

eecc 6y ago

Perhaps USians as desensitized to guns but I took pause when reading that the police officers drew their guns when chasing the fugitive in the original incident, about shoplifting candies from a hotel lounge? (And I wonder if the concierge that called the cops on this guy hadn’t provoked a taunt with some “kind words”.)

renewiltord 6y ago
Americans are, as a rule, highly authoritarian. You'll see it online as people use pseudo-bureaucratic corpo-speak to justify state violence: "refusal to obey a lawful order", "did not comply with a lawful request", "in violation of state regulation" etc. Having lived elsewhere and in the US (a country I practically adore), I suspect it is because Americans have not really had true authoritarianism in any sense at any point in their history. Every time they stray close to the darkness and walk away unscathed and it convinces them that they can never really be consumed by it so they're willing to walk its very edge. Perhaps the UK having a monarch and a state religion is a visible reminder of what a monstrosity the state can be. It's funny that both nations have populations that support the massive expansion of the state into human affairs: it's just that the Americans want it to enforce behaviour and the British want it to redistribute wealth. The most amusing manifestation of the American comfort with violence is the difference in how streakers are handled in British football games and American football games. The former have overweight stewards lumber after naked people, failing repeatedly to apprehend, eventually accompanying them off the field. The latter have large, muscular guards catch up to and violently tackle the streaker to the ground.
- watwut 6y ago
  > Americans have not really had true authoritarianism in any sense at any point in their histor That is not really true. American history is constant struggle between authoritarianism and opposition.

banjomet 6y ago

"The software, which was created by Clearview Al, was criticized for its heavy reliance on billions of social media photos to identify criminal suspects." So how exactly did they get access to all photos? It must have been trained on public networks like instagram, not facebook, right?!? If it was FB that would be interesting.

Cactus2018 6y ago
https://www.theverge.com/2020/2/6/21126063/facebook-clearview-ai-image-scraping-facial-recognition-database-terms-of-service-twitter-youtube https://www.theverge.com/2020/2/6/21126063/facebook-clearvie... > "Facebook and LinkedIn are latest to demand Clearview stop scraping images for facial recognition tech. Twitter and YouTube have also objected." https://slate.com/technology/2020/02/youtube-linkedin-and-others-serve-clearview-ai-with-cease-and-desist-letters.html https://slate.com/technology/2020/02/youtube-linkedin-and-ot... > "Facebook notably has not sent a formal cease-and-desist letter but claims to have sent other letters to Clearview to request more detail on its practices and then eventually “demanded” that it stop scraping user data. Peter Thiel, a venture capitalist and notable surveillance enthusiast who sits on Facebook’s board of directors, invested $200,000 in Clearview’s first round of funding."

upofadown 6y ago

>...notified Woodbridge police they had a “high profile” match to the photo,... That's weird. You would think a facial recognition system would return hundreds of matches in a case where it was searching through a whole lot of images. What does "high profile" mean in this case?

drak0n1c 6y ago

In Shenzhen, starting in the last couple years if you jaywalk your face is automatically recognized and the fine is automatically deducted from your bank account. It's extremely accurate due to full access to corroborating location/transit/payment data. This happens within 20 seconds, and even includes expats. Your face was permanently indexed during the passport check. Such an automated system at first glance may seem like it reduces on-the-ground human flaws and bias, but it actually enables far deeper corruption. Those with administrative control would be able to selectively highlight or concoct any sort of offense to undermine their personal or political opponents. "Show me the man and I'll show you the crime" - Lavrentiy Beria, Stalin's Chief of Secret Police

daniellarusso 6y ago
Does this work with sunglasses and facemasks?
- pmiller2 6y ago
  Facial recognition can work with ordinary facemasks. I don't have a link handy, but I'm sure Google can produce one, given the timeliness of the question. As to your question, I also wonder about such things as: * Makeup: https://www.vogue.com/article/anti-surveillance-makeup-cv-dazzle-protest https://www.vogue.com/article/anti-surveillance-makeup-cv-da... * Hair styles: https://cvdazzle.com/ https://cvdazzle.com/ * Clothes: https://www.businessinsider.com/clothes-accessories-that-outsmart-facial-recognition-tech-2019-10#belgian-scientists-developed-a-prototype-for-a-graphic-print-that-could-be-added-to-clothing-to-attack-and-baffle-surveillance-technology-11 https://www.businessinsider.com/clothes-accessories-that-out... * Infrared LEDs: https://www.schneier.com/blog/archives/2018/03/fooling_face_re.html https://www.schneier.com/blog/archives/2018/03/fooling_face_... * Image manipulation software: https://www.theverge.com/2020/8/4/21353810/facial-recognition-block-ai-selfie-cloaking-fawkes https://www.theverge.com/2020/8/4/21353810/facial-recognitio... And other things.
  - Retric 6y ago
    The first several links all say they make a big difference. Face masks are already known to stop the spread of coronavirus. Apparently, they can also make it much harder for facial-recognition software to identify you, too. (cnn) It’s true accurate identification in ideal conditions is still possible, but catching random jaywalkers is hardly ideal conditions.

11thEarlOfMar 6y ago

It was about 10 days after the incident that he was in custody. What did the 3 eye witnesses say? Did they make positive identifications? Did the officers involved in the initial incident claim he was the same person? Were any of the 7 officers the same ones who encountered him in the hotel? How about the hotel staffer? Did they make a positive ID? Was 'Jamal' a hotel guest? Seems like there should be a discussion about all of this in the article. Facial recognition as the sole evidence must not be used to make arrests.

wvenable 6y ago
Eye witness identifications are very likely worse than facial recognition. There should be some other evidence than either of those.

agnosticmantis 6y ago

Are the creators of these algorithms penalized for their false positives? I think the confidence levels returned by these algorithms would be quickly calibrated if there was some strict penalty for high-confidence false positives.

KittenInABox 6y ago
I would prefer that there is a strict penalty for people who rely too much on their tool that they can't even do their jobs correctly. I'd fire a carpenter if an AI told the carpenter to fuck up my cabinet, because I expect my carpenter to double-check. Similarly if the carpenter finds the AI consistently useless and an avenue to get their ass sued, then they'll decline to purchase shit AI.
- agnosticmantis 6y ago
  I agree. The penalty on the user of AI would in turn incentivize the user to penalize/reward the AI developer. But I’m not sure if it’s reasonable to require every user to be able to assess the long run accuracy of a statistical system given their limited interaction. Maybe there’s a market for companies to reliably assess and certify AI systems or by creating a sort of a prediction market that penalize overconfident false positives.
- deleted 6y ago
  [deleted]
deleted 6y ago
[deleted]
LanceH 6y ago
Why should they be? This system is saying, "of the 10 million people we have in our database, this one looks like the photo you gave us." It is the police and prosecutor who are asserting, "this person did it and should be held in jail." This tool is a visual equivalent of looking up someone's name and is the visual equivalent of the no-fly-list being used as though no two people share the same name.

okthenletssee 6y ago

Is it really so uncommon for suspects of a crime to spend time in jail? I think I have read such stories without computers having anything to do with it. It's just a sensationalist spin trying to ride the AI scare.

diebeforei485 6y ago

What judge signed the arrest warrant, and why are they still a judge?

DubiousPusher 6y ago

It seems to me this is not a facial recognition story but a story about how the state, with very little evidence, can ruin your life and leave you with very little recourse. As concerned as I am about state use of facial recognition, these kind of things have been happening to people for decades now. The solution to this problem is to better secure the rights of people accused of crimes. We need to ensure that people accused do not lose their jobs and homes. That they have fair opportunity to communicate with counsel and family (both are often needlessly limited by DoC). And the state must be responsible to mitigate its mistakes. Bail requirements are too often granted as par for the course and need a "burden of proof" type rethinking. Holding people at home should be required before incarceration. Lastly we need to remove the power of the legislatures to unequally empower prosecutors and public defense attorneys.

js2 6y ago
> a story about how the state, with very little evidence, can ruin your life and leave you with very little recourse. Buttle? Tuttle? https://vimeo.com/224785800 https://vimeo.com/224785800
Blikkentrekker 6y ago
> It seems to me this is not a facial recognition story but a story about how the state, with very little evidence, can ruin your life and leave you with very little recourse. The facial recognition is relevant, as they relied on facial recognition software, though it is apparently illegal in New Jersey. From the article as well: > He asked for a lawyer, then was taken to a hallway where he was handcuffed to a bench. About an hour later, the officers – he counted seven – told him they were going to take him to a different room for more questions. As far as I understand U.S.A. law, when a suspect ask for a lawyer, the interrogation is to stop immediately, and a lawyer must be provided or he must be allowed to call one, and the lawyer must be præsent ere the interrogation continue. The real takeaway seems to be that the police did something illegal. The rest of your post is about many more rules, but the issue is that the rules were broken here.
- tinalumfoil 6y ago

renewiltord 6y ago

The problem appears to be that police and prosecutors are either morons or incentivized so strongly to arrest and convict that they ignore obvious nonsense. Given that this is the case, the use of AI cannot be accepted by any members of the US justice system. If I had access to a facial recognition system, I would use it as a whittling down tool, to reduce the amount of hay to find the needle in. But they clearly see a straw and conclude that it is a needle. It's like giving a four year old a handgun. It's just not responsible. They're just not as good at this as I am. So no handgun for you. It's a pity. Ideally you should be able to discriminate, and intelligent police should be given the tool, and stupid police not. However, because stupid police and smart police exist in an emulsion, it is better to just waste the time of the smart police than to give the stupid police too powerful a tool.

MeinBlutIstBlau 6y ago
When the state doesn't have to foot the bill taking you to court, of course they don't care. If the procsecutor had to pay all your legal fees if they lost, we'd either get some pretty amazing state prosecutors or ones who who only accept evidence from cops that can convince a judge. It's almost as if you stop cops from just locking people up it creates an immense snowball effect on the whole system...
tolbish 6y ago
> If I had access to a facial recognition system, I would use it as a whittling down tool, to reduce the amount of hay to find the needle in. But they clearly see a straw and conclude that it is a needle. It's like giving a four year old a handgun. It's just not responsible. They're just not as good at this as I am. So no handgun for you. Just wanted to say I've never heard such an argument worded so well. What is the course of action for the common man other than verbalizing such thoughts to local city councils and law enforcement?
- renewiltord 6y ago
  I think the local powers have figured out the route that will already succeed: blame the tool. That way permits everyone to leave with their egos intact. The Art of War 7.36 in action, one might say.

flatus 6y ago

That's nothing: Steven Talley was identified by the FBI as the primary suspect in two bank robberies using a facial recognition algorithm. He had an iron-clad alibi, but the police and FBI weren't convinced. In court one of the bank tellers said Talley definitely wasn't the robber. Nonetheless Talley lost his job, his wife and his family and was held in prison for months: "LOSING FACE: How a Facial Recognition Mismatch Can Ruin Your Life" https://theintercept.com/2016/10/13/how-a-facial-recognition-mismatch-can-ruin-your-life/ https://theintercept.com/2016/10/13/how-a-facial-recognition... FTFA: "Talley said he was held for nearly two months in a maximum security pod and was released only after his public defender obtained his employer’s surveillance records. In a time-stamped audio recording from 11:12 a.m. on the day of the May robbery, Talley could be heard at his desk trying to sell mutual funds to a potential client." Today Talley is still trying to claw his way back to normalcy. In the outstanding book "Hello World" Hannah Fry examines Talley's story as part of a chapter on crime, AI and facial recognition. Reading Fry's book convinced me that facial recognition software simply does not work well enough to use in police work. As Fry says: "If you're searching for a particular criminal in digital line-up of millions...the best-case scenario is that you won't find the right person one in six times...". That is not nearly good enough for law enforcement and the courts. "Hello World" by Hannah Fry https://www.amazon.com/Hello-World-Hannah-Fry/dp/0857525255 https://www.amazon.com/Hello-World-Hannah-Fry/dp/0857525255

vrperson 6y ago
Again, how is that story a failure of AI/face recognition? At some point they even have a human compare the likeness, and the human also concludes it is the same person. The article even features the sentence "teve Talley is hardly the first person to be arrested for the errors of a forensic evaluation." And yet people seem to be hellbent to make it about AI.
- mahkeiro 6y ago
  It’s a failure in the sense that people put too much confidence in this kind of algorithm and put them above any eye witness when they should just be considered as another element.

duxup 6y ago

What happens if this went to trial (let's say there was enough other evidence)? How do you question your accuser? Do you have access to the software? How much access? Do many people even have the resources to examine the software?

pacamara619 6y ago

"Software problem, nothing we could do." Every time someone fucks up using technology and noone is held accountable, that's what we hear.