15 ms·
Apple sued by teen wrongly accused of shoplifting by unreliable facial-rec tech
- deleted 5y ago[deleted]
- handmodel 5y ago>identified the individual as Ousmane Bah based on the examination of the temporary learner's permit he is said to have had on him So essentially a kid stole another kids ID. Because the (thieving) kid had gotten in trouble with the same ID before it was also a match in the database. It does seem like a true failure but the headline makes it sound like this was an AI falsely accusing someone based on pure randomness. Even without the tech, the police would have had the name of the wrongly accused person because the security guard only had the fake/wrong ID and the person's ability to match faces was not great.
- threatofrain 5y agoNot just another kid, but a friend. Ousmane Bah is an honors student, while Mamadou Barrie is potentially responsible for a string of thefts and the impersonation of his friend to the police. > "The arresting officer was able to identify the impostor as Mamadou Barrie, a friend of the Plaintiff, who apparently stole the learner’s permit from the Plaintiff," the complaint says. "These arrests specifically [noted] that Barrie had pretended to be Ousmane Bah." > The detective subsequently submitted an information request "to the NYPD’s Facial Identification Section (FIS), which identified the photograph as potentially depicting two people, one of whom was purportedly Ousmane Bah – and the other was the actual thief, Mamadou Barrie."
- pishpash 5y agoThat's why you pick your "friends" carefully.
- deleted 5y ago[deleted]
- naikrovek 5y ago> the headline makes it sound like this was an AI falsely accusing someone based on pure randomness. no, it doesn't. the headline makes it sound like facial recognition is unreliable for some reason. the reason, in this case, is that the Apple store and/or SIS personnel gave the recognition system Ousmane's name when they weren't able to prove it was him, and the document provided for identification was not actual proof of identification. Further facial recognition matches assigned further blame to the misidentified person, increasing the evidence against him. Then, because store owners and security firms really want to put away shoplifters, they claimed that video proof of the shoplifting was routinely deleted when there was no policy describing a deletion policy within Apple or SIS, only for video proof that the individual was not Mr. Bah to later be found. in this case, facial recognition tech was unreliable entirely because of the people running it.
- frogpelt 5y agoThe HN title is bad. The teen was not falsely accused by unreliable facial recognition technology. I sort of skipped through the actual headline trying to figure out how AI was involved. It was. But only after several blunders by people.
- elliekelly 5y agoIf the teen was falsely accused and the false accusations came from the facial recognition technology then the facial recognition technology was, in fact, unreliable.
- mattmcknight 5y agoThe false accusations didn't come from the facial recognition technology. They came from believing the criminal using this falsely accused person's ID card which did not have a picture on it. Facial recognition tech correctly matched the face of the criminal to the other video of the criminal.
- 411111111111111 5y ago
- mannykannot 5y agoYou seem right to say it is not a case of AI falsely accusing someone based on pure randomness, but that is no reason to dismiss what happened here. The most troubling aspects are that SIS added what amounts to false information to its databases, fabricated multiple false cases against Bah, claimed to have evidence it did not have, used it to detain him and hand him over to the Paramus police for arrest (who failed in their own due diligence, partly on account of this false information), and then attempted to hide what it had done, including, apparently, destroying evidence - evidence against itself, not Bah.
- handmodel 5y agoIf a DNA test identified the wrong suspect because someone mixed up the names when entering them into a database I wouldn't say that there was "unreliable DNA tech". Maybe other people didn't read the headline as I did. But if I hadn't bothered to click on it I 100% was just going to assume it was because the tech itself was unreliable and they picked out someone who looked like him. Perhaps if I wasn't read up on the debate around this I wouldn't have thought that - but I imagine most people would assume this with confidence if they saw the headline on their feed.
- mannykannot 5y agoYou are making a valid, but relatively small, point about how the article is written. People who think they have learned something important from a headline without verifying what they think they have learned by applying critical analysis to the article are likely to be misinformed in all sorts of ways regardless of what they do read. Note that we are both talking about other people - we, of course, are not like that!
- deleted 5y ago[deleted]
- elliekelly 5y agoThe AI falsely accused someone because the AI was fed garbage data. If you’re going to build an algorithm that can levy such serious charges against people it certainly seems negligent to create a process that allows random unverified training data to be input into the system. There are real people who suffer real consequences because of the programs we write. When things go wrong we don’t get to point the finger at the algorithm and the users and take no blame as the developers.
- dracodoc 5y agoThe headline is accurate because that's one subject of the lawsuit. If I'm understanding it correctly after skimming through the filing(read especially page 6 and 9): 1. the plaintiff argues that in Boston case (page 6) apple store only took less than 10 minutes to identify the shoplifter, so they probably used facial recognition tech. Note the shoplifter left the store, apple found the incident, report it to police within 10 minutes, without actually detain the person. 2. I'm guessing the facial recognition is actually accurate because it was based on their previous security video, not plaintiff's real id photo(which apple probably don't have access all the time). So the facial recognition just linked one person with previous security video properly, and the id of person was based on previous record made by employee. 3. Thus there is nothing wrong here with facial recognition, but the title will definitely evoke fear and people's emotion since most people will not see the details.
- MeinBlutIstBlau 5y agoHope that kid wins so much money he can just drop out of college now. This is such blatant malicious activity on both these companies parts. Deleting video evidence and confirming with police based on the acquisition of a permit not allowable for identification. If you can't even use the ID to vote, why do you get to use it to incriminate someone? This whole sub-contracting policing out to private companies is going to end up being a constant state of witch trials. At least the state is the law. These two companies aren't in the slightest.
- judge2020 5y ago> If you can't even use the ID to vote, why do you get to use it to incriminate someone? Because there's extra requirements for voting that aren't related to actually having a government-issued ID card.
- deleted 5y ago[deleted]
- protomyth 5y agoThis was not a government issued ID card as it says "This temporary document is not to be used for identification purposes."
- judge2020 5y agoThis doesn't address how being able to vote has anything to do with it. You can't vote with the picture-included permanent learner's permit or a real driving license in New York either if you're under a certain age.
- deleted 5y ago[deleted]
- blunte 5y agoYour answer doesn't address the question: "why do you get to use it to incriminate someone?" But the point remains: if an ID card isn't considered valid for voting, then why should it be considered valid for identifying an individual for a crime? It is a valid ID (identification record) or it is not.
- deregulateMed 5y ago"Privacy"- Apple commercial, black background and white text
- sumthinprofound 5y ago"Apple and SIS are said to have told the New York court that neither firm has any written policy on video retention." Why would there not be a written video retention policy?
- marcinzm 5y agoBecause then they'd not be able to delete it as freely to avoid losing a lawsuit.
- deleted 5y ago[deleted]
- judge2020 5y agoNote that the Connecticut learner's permit does have a photo ID on it - the article gets it wrong, as the complaint states that the learner's permit he had was actually a New York learner's permit[0]. However, on that note, does anyone have a photo of such permit? All Google images of 'learners permit new york' have photo IDs; the closest thing I can find is the post-test temporary full license[1], which doesn't include information like the person's height. The exhibit is also restricted on Pacer[2], likely because it contains PII. 0: https://regmedia.co.uk/2021/05/29/pacer_bah_apple.pdf#page=3 https://regmedia.co.uk/2021/05/29/pacer_bah_apple.pdf#page=3 1: http://2.bp.blogspot.com/-c7SAttW166s/VaOtXtrptTI/AAAAAAAABcY/3tasD5y68nI/s1600/Driving%2Btest%2Breceipt.jpg http://2.bp.blogspot.com/-c7SAttW166s/VaOtXtrptTI/AAAAAAAABc... (linked from this public blog http://invisiblevisibleman.blogspot.com/2015/07/a-driving-test-mistaken-questions-and.html http://invisiblevisibleman.blogspot.com/2015/07/a-driving-te... ) 2: https://i.judge.sh/similar/Dash/chrome_fY4ObopfUE.png https://i.judge.sh/similar/Dash/chrome_fY4ObopfUE.png
- marcinzm 5y ago>However, on that note, does anyone have a photo of such permit? I have a temporary non-learners license but due to PII I'm not taking a photo of it. It is a printout that contains everything except a photo (so name, height, eye color, etc.). It's used as a temporary license until the real one is mailed to your address. This is distinct from the post-test printout which would be, I'm guessing, used in conjunction with a learners permit license. The temporary license printout is used on its own without another permit which is why it contains more info.
- deleted 5y ago[deleted]
- exporectomy 5y agoIt gets worse - "a different individual with the same name "Ousmane Bah," this one a resident of Willingboro, New Jersey, showed up for the summons. He was not the thief, the complaint says"
- yepthatsreality 5y agoYou’re face-ing it wrong!
- mgleason_3 5y agoIf I understand the story correctly, they had the suspect in custody and yet failed to positively ID him, relying on an ID he had stolen from an innocent man named Ousmane Bah. The ID had no photo and was labeled "not to be used for identification". SIS filed a police report which said they had video evidence showing the perpetrator. They then put out a "BOLO" with the innocent man's name, had him arrested, figured out they have the wrong guy, do nothing about it and had him arrested again. When Bah appeared in court to answer charges and Apple was asked to present the video that would have cleared Bah, Apple claimed that the video had been deleted. The video was later found by Bah's attorney during discovery. This happened more than once. A warrant was issued for Bah’s arrest and contained the photo of the imposter. Bah was arrested even though the he doesn't resemble the imposter, other than being Black. Prosecution against Bah continued in multiple states through June 2019. Hmmm, seems like a lot of people involved in this case should be fired, including the Apple Employees, SIS Employees and Police officers, reguardless of whatever other outcomes there are.
- ashtonkem 5y agoSeems like some Apple employees need to be thrown in jail for lying to the court.
- encryptluks2 5y agoLOL, if only justice worked that way.
- tzmudzin 5y agoWell, maybe it should. I'm not in favor of handing out prison sentences, but that's probably what the victim faced. Eye-for-eye is a bad strategy, but no consequences for such accusations is not a solution.
- jeltz 5y agoOn paper it does, what they did is obviously criminal under current laws. The issue is that these crimes are almost never prosecuted.
- trhway 5y ago>Apple and SIS have a qualified law enforcement privilege that allows them to err in store security-related accusations and not be sued for it. that is the gist. They don't care because they can do all that stuff and can't be held responsible.
- pbadenski 5y agoI'm a bit speechless... Can someone explain how does this scheme can even exist? It sounds preposterous.
- trhway 5y agoApple chief of security was recently indicted on a charge of a bribe to Santa Clara county sheriff, and the case was dismissed last month. There are 3006 counties in US, and one can let the imagination run wild :)
- 8ytecoder 5y agoI don't want to take away anything from your argument but that case is a bad example. There was no evidence to show corrupt evidence and it was dismissed as such. https://www.mercurynews.com/2021/06/01/santa-clara-county-judge-tosses-concealed-gun-permit-bribery-charge-against-apple-exec/ https://www.mercurynews.com/2021/06/01/santa-clara-county-ju...
- CAX 5y agoThree years ago I asked facebook GSOC to look into why someone driving a muscle car with plates reading 'CA X' registered to an owner in San Jose, sideswiped me immediately in front of the main facebook site, main building, near the sign, while he (a Pakistani looking man?) drove erratically out of the entrance while making a left onto the highway and stopped at the red light facing the bridge. I pulled up beside him and told him to pull over and then he sped off down university. I chased him through two red lights down the avenue before the police office on the phone (I believe Menlo PD) told me I would have to file a report at the Menlo PD station, and they are not going to help me pursue him in the moment that evening but she confirmed the plates were out of San Jose. I did not have time to do as contractor being used and abused by h1-b girlbosses and insane Korean youths on the team at FB. Instead of going to the police station I contacted GSOC at facebook. I followed up two weeks later and my contact on the request I had escalated to GSOC had been fired. Two weeks later I was also fired after getting an email about installing teamviewer on my device, which made me a spy Or something. Needless to say, a bad experience as a contingent workforce for facebook, who seem to hire really unimpressive young people -- such as myself -- to the point that it was raising a lot of eyebrows on my team at the time. Something needs to be done about these guys, they are up to no good and causing lots of trouble in the neighborhood. Also the facebook employees dont like it when you ask the 'building operations' team to keep the bathrooms clean and they attack you for being white when you expect the bathroom sink to not be soaking wet every time you walk up to wash your hands -- I recall comments along the lines of "arrogant [sic] white man" etc etc for relaying a message to the building team about the constantly wet sinks. This is why I believe that these GSOC teams are probably papering over lots of bad behavior, and they are just Pinkerton agents at best
- belltaco 5y ago>Apple then told the Suffolk County prosecutor "that the video evidence of the impostor, which would have completely exculpated Ousmane Bah, had been routinely deleted." Why would anyone routinely delete evidence of crime instead of making a copy of it before it was deleted? What's the point of having surveillance equipment and footage if it's just routinely deleted even if there was a theft? >The video from an October 2018 theft misattributed to Bah in Rockaway, New Jersey, was also deleted. Oh wow, again? >And as it turned out, the video of the Boston incident turned up eventually – Bah's attorneys found it during the discovery process. It showed the impostor, not Bah. They should open a criminal investigation into what happened with the video. Intentionally accusing someone they know to be innocent, taking their time and freedom is completely inexcusable.
- pessimizer 5y agoBusinesses (at least in the US) try to write the shortest possible data retention policies possible so that evidence is hopefully deleted before any litigation begins. If you had a written policy and you hadn't received a court order before deleting, you've won.
- ashtonkem 5y agoI think there should be a carve out for evidence provided to the state or the defense. If you provide a sworn statement that evidence exists and then destroy it before the court is done with it, you should face contempt of court charges.
- tomxor 5y ago> Why would anyone routinely delete evidence of crime instead of making a copy of it before it was deleted? (speculation) because the prosecutors preferred to ruin an innocent man's life over appearing incompetent... it technically risks evidence tampering, but they can probably just argue it back to incompetence so they have nothing to lose.
- rtpg 5y agoThe beauty of an adversarial system, where the prosecution's goal is to win, not to find the truth.
- ibudiallo 5y agoI've been having this discussion a lot lately. Ok, let's say a crime occured and you have a hard time identifying the suspect. A colleague of yours in the precinct is walking over and sees the picture on your screen. "Wait a minute, that face looks familiar." He goes to the database and indeed sees a face that looks close enough. Now what is the next step? You don't toss the person in the police van and celebrate the conviction. Instead, you investigate. "Where we're you that evening? do you own a outfit matching the suspect? Etc." This kid would have been quickly cleared. It's already terrifying to be questioned by the police when you are innocent. How do you think he felt when they said they had video evidence of him shoplifting. They already made up their mind. Now replace your colleagues with an AI. Simply because you have a match does not mean you have the suspect. Just over a year ago we were discussing the same issue: https://news.ycombinator.com/item?id=23628394 https://news.ycombinator.com/item?id=23628394
- tolbish 5y ago> A colleague of yours in the precinct is walking over and sees the picture on your screen. "Wait a minute, that face looks familiar." He goes to the database and indeed sees a face that looks close enough. Now what is the next step? That depends on if the goal of those using the technology is arriving at the truth or if the goal is naming a suspect who you can reliably stick a guilty verdict on.
- NickNameNick 5y agoI shared an office space with a company that provided video analysis and recognition as a commercial expert witness service. (This was in The UK). They had a system of levels of confidence they would use in their reports when identifying someone. The highest level of confidence was achieved when a close friend or family member identified the person in the picture of video. There were progressively lesser levels of confidence for people less close to the purported subject of the image. They also had a system of points of similarity/difference. but they normally only used that to establish that the same subject had been observed by multiple cameras, rather than to establish the identity of the subject.
- sneak 5y ago> Now what is the next step? You don't toss the person in the police van and celebrate the conviction. Instead, you investigate. You overestimate the initiative and competence of the police.
- warent 5y agoApple and SIS have a qualified law enforcement privilege that allows them to err in store security-related accusations and not be sued for it. However, if they exhibit "reckless disregard for the truth" – ignoring obvious facts, for example, they lose that privilege. Curious to wonder if they mean they just lose that privilege on a case-by-case basis, or if they lose it for all future errors in general. If it's the latter, then Apple is in serious trouble and they'll probably end up paying this kid big time in a sealed settlement so they don't lose this privilege.
- cable2600 5y agoLoss Prevention is a very hard job.
- bell-cot 5y agoIf done with this level of competence and honesty, what jobs (if any) would you NOT describe as "very hard"?
- cable2600 5y agoJust that LP has rules like you can't get physical with them and hit them, you can't use a weapon, if they have a weapon let them go. You are not a police officer or security officer, you are a store employee or contractor that is disposable should an incident happen.
- dschuetz 5y agoNot only is it obvious that the technology is not working, but this also shows blatant incompetency on part of users of this tech. This no longer counts as "the algorithm is flawed". The whole system is wrong.
- hemantv 5y agoWhy can't every small business have this qualified immunity How do I also get qualified immunity Or is it dependent on how much money and influence you have? So money can buy qualified immunity that's pretty sweet deal.
- deleted 5y ago[deleted]
- anigbrowl 5y agoRun a security company, contract with public officials, you're most of the way there. Government can be a powerful social technology at best, but at worst it's just a protection racket.
- otterley 5y agoEvery store operator in the U.S. has the so-called "shopkeeper's privilege" with respect to suspected shoplifters. It's not limited to Apple.
- gtoast 5y ago"The lawsuit contends that Apple and SIS exhibited reckless disregard for the truth by misidentifying Bah as the perpetrator of multiple shoplifting crimes at iStores..." What the heck are "iStores"?
- raunak 5y agoThe article jokingly refers to many things as i(Anything). iGiants in the beginning, for example. It’s an odd, and poorly executed attempt at humor/fitting in, but that’s what it is
- ngc248 5y agoIn the article they may have used it for humorous effect, but in India, apple products are actually sold in iStores. https://9to5mac.com/wp-content/uploads/sites/6/2013/03/istore-reliance.jpg https://9to5mac.com/wp-content/uploads/sites/6/2013/03/istor...
- deleted 5y ago[deleted]
- YeBanKo 5y agoI am fascinated by this subject, because of how scary misapplication of AI can be. But this has very little to do with AI/facial recognition tech itself. The name was picked up from a lost learner's drivers permit, that was found on an suspect while in custody, the permit did not have a photo and then this stolen name had been propagated further and attached to other cases. They used a paper without a photo as identification. It actually says right on it, that it is not to be used as identification, WTF? > In addition, it asserts Bah's apprehension was in part due to the application of unreliable facial-recognition technology in the shoplifting incidents in New York. Vague statement, unclear what exactly role of the facial recognition was. > The detective subsequently submitted an information request "to the NYPD’s Facial Identification Section (FIS), which identified the photograph as potentially depicting two people, one of whom was purportedly Ousmane Bah – and the other was the actual thief, Mamadou Barrie." Seems like the software did what it had to: identified two people. The article also says that they were friends, so it is not surprising that were both present on some frames. It can be source of some confusion as well, not tech related though. > The video from an October 2018 theft misattributed to Bah in Rockaway, New Jersey, was also deleted. Apple and SIS are said to have told the New York court that neither firm has any written policy on video retention. Yeah, this is fishy. I am very doubtful they don't have any written retention policy. Human error, eh?
- ksec 5y ago>>Apple and SIS have a qualified law enforcement privilege that allows them to err in store security-related accusations and not be sued for it. This pretty much sums up modern days Apple. In terms of above the law, being righteous. And it wasn't a honest mistake or one-off incidents. They have been doing it repeatedly. It just reeks of hypocrisy.
- 1cvmask 5y agoOusmane Bah was a black man (with a Muslim name). Would Apple have acted otherwise if he wasn't? I wonder if there are such datapoints of clearly wrongfully accused white people where they still had a BOLO put on them despite no evidence.
- blueflow 5y agoAh, the US, the only place in the world where judging a social situation by the skin colors of the participants is not just acceptable, but also the way seen as most virtuous.
- 1cvmask 5y agoApple is the company that talks about being virtuous. You can even say that they are even sanctimonious. Just look at Tim Cook's speeches on virtue, privacy and tracking. Yet their actions, based on their own framework, here are the opposite of their own wording. They fired employees for "lesser" offenses.
- blueflow 5y agoI was not talking about Apple, i was talking about you.
- Niglodonicus 5y agoMaybe, maybe because pointing out such a thing is evidence of a bigger problem (systemic discrimination & racial bias) that has been demonstrated to not be only exclusive to the realm of human investigators, but also in a lot of facial recognition software (that both trains on a disproportionate amount of black faces and is poorly able to differentiate between distinct black faces).
- blueflow 5y agoFrom what i know about the US, racism is one of your lesser issues. Its just very, very loudly overemphasized because it distracts from the failings of your social, healthcare and governmental system. The poor people suffer the most from this, and take a guess who is disproportionately poor.
- kitsune_ 5y ago"The bureaucratic mentality is the only constant in the universe." — Dr. Leonard H. McCoy, Star Trek IV: The Voyage Home.
- deleted 5y ago[deleted]
- danaris 5y agoThis appears to be an update to a story that was discussed here on HN over 2 years ago: https://news.ycombinator.com/item?id=19726528 https://news.ycombinator.com/item?id=19726528 Now, as then, facial recognition is being used as clickbait, when all that really happened is that this guy stole an ID, got caught shoplifting in an Apple Store with that ID, and thus had the information on the ID incorrectly associated with his face—either in a facial recognition database, or simply by the photo being used with this stolen ID. My understanding from the previous story was that it was not Apple that took the information from the ID, but rather the police, who gave that information to Apple. It looks from this story like there's also a third party involved, SIS, the security contractor, which, of course, further muddies the waters.
- deleted 5y ago[deleted]
- Lapsa 5y agograb that money!
- prince005 5y agoIs it just me or a part of Apple is just rotting away?
- mickhead23 5y agoPrivate security guards can apprehend and cuff a person? I live in the US and have never heard this before.
- josephcsible 5y agohttps://en.wikipedia.org/wiki/Shopkeeper%27s_privilege https://en.wikipedia.org/wiki/Shopkeeper%27s_privilege
- caseysoftware 5y agoA mistake is one thing, subverting safeguards and deleting data is something else entirely. The developer who developed the system to falsify EPA results for Volkswagen went to jail for 3 years. The judge specifically cited that he had a professional duty to not commit illegal behavior but chose not to because he didn't want to lose his job. Ref: https://www.nytimes.com/2017/08/25/business/volkswagen-engineer-prison-diesel-cheating.html https://www.nytimes.com/2017/08/25/business/volkswagen-engin... Based on how key software is in so many things, we're going to be faced with a choice soon: a) As an industry, clean house and hold ourselves accountable to better practices, uncovering and fixing honest mistakes, and punishing misbehavior; OR b) Do nothing and let regulators write the rules for us.. which we know will be outsources to the big consultancies like Deloitte, BAH, PwC, KPMG, etc. If we don't all get onboard for A, the alternative will be ugly.
- mjs7231 5y ago> Apple and SIS have a qualified law enforcement privilege that allows them to err in store security-related accusations and not be sued for it. Can someone smarter than me explain what this line is referring to? Sounds scary.