5 ms·

>identified the individual as Ousmane Bah based on the examination of the temporary learner's permit he is said to have had on him So essentially a kid stole a

>identified the individual as Ousmane Bah based on the examination of the temporary learner's permit he is said to have had on him So essentially a kid stole another kids ID. Because the (thieving) kid had gotten in trouble with the same ID before it was also a match in the database. It does seem like a true failure but the headline makes it sound like this was an AI falsely accusing someone based on pure randomness. Even without the tech, the police would have had the name of the wrongly accused person because the security guard only had the fake/wrong ID and the person's ability to match faces was not great.

threatofrain 5y ago
Not just another kid, but a friend. Ousmane Bah is an honors student, while Mamadou Barrie is potentially responsible for a string of thefts and the impersonation of his friend to the police. > "The arresting officer was able to identify the impostor as Mamadou Barrie, a friend of the Plaintiff, who apparently stole the learner’s permit from the Plaintiff," the complaint says. "These arrests specifically [noted] that Barrie had pretended to be Ousmane Bah." > The detective subsequently submitted an information request "to the NYPD’s Facial Identification Section (FIS), which identified the photograph as potentially depicting two people, one of whom was purportedly Ousmane Bah – and the other was the actual thief, Mamadou Barrie."
- pishpash 5y ago
  That's why you pick your "friends" carefully.
  - deleted 5y ago
    [deleted]
naikrovek 5y ago
> the headline makes it sound like this was an AI falsely accusing someone based on pure randomness. no, it doesn't. the headline makes it sound like facial recognition is unreliable for some reason. the reason, in this case, is that the Apple store and/or SIS personnel gave the recognition system Ousmane's name when they weren't able to prove it was him, and the document provided for identification was not actual proof of identification. Further facial recognition matches assigned further blame to the misidentified person, increasing the evidence against him. Then, because store owners and security firms really want to put away shoplifters, they claimed that video proof of the shoplifting was routinely deleted when there was no policy describing a deletion policy within Apple or SIS, only for video proof that the individual was not Mr. Bah to later be found. in this case, facial recognition tech was unreliable entirely because of the people running it.
- frogpelt 5y ago
  The HN title is bad. The teen was not falsely accused by unreliable facial recognition technology. I sort of skipped through the actual headline trying to figure out how AI was involved. It was. But only after several blunders by people.
  - elliekelly 5y ago
    If the teen was falsely accused and the false accusations came from the facial recognition technology then the facial recognition technology was, in fact, unreliable.
    - mattmcknight 5y ago
      The false accusations didn't come from the facial recognition technology. They came from believing the criminal using this falsely accused person's ID card which did not have a picture on it. Facial recognition tech correctly matched the face of the criminal to the other video of the criminal.
      - 411111111111111 5y ago
        Doesn't that make it worse though? Drum up false charges and they get automagically mapped to the face with nobody able to undo anything. They're now forever a criminal. Honestly, it sounds extremely dangerous in a field such as criminal investigations.
        zo1 5y ago
        This is precisely why the "fear" or rather stigma around facial-recognition needs to go away. If it weren't there, it'd have been expected and granted that the government's job is to have valid, verified and correct biometric information on the citizen (because they're generally the ultimate/final custodian of people's identity), and the facial-recognition would have to match against that verified data, instead of some random video or fake ID. So instead we have scattered silos of partial and probably incorrect data in various places. FBI has a bit, Police department in state X has one, State Y's DMV has something wrong because X years ago this person was a victim of identity fraud, etc etc. It's a giant mess and no wonder the police and various entities in this example failed utterly. They're ultimately all victims of a system that has no single source of truth, yet we're tasking them with using "tech" to solve crime. So instead of fixing the source of truth, we blame the tech.
mannykannot 5y ago
You seem right to say it is not a case of AI falsely accusing someone based on pure randomness, but that is no reason to dismiss what happened here. The most troubling aspects are that SIS added what amounts to false information to its databases, fabricated multiple false cases against Bah, claimed to have evidence it did not have, used it to detain him and hand him over to the Paramus police for arrest (who failed in their own due diligence, partly on account of this false information), and then attempted to hide what it had done, including, apparently, destroying evidence - evidence against itself, not Bah.
- handmodel 5y ago
  If a DNA test identified the wrong suspect because someone mixed up the names when entering them into a database I wouldn't say that there was "unreliable DNA tech". Maybe other people didn't read the headline as I did. But if I hadn't bothered to click on it I 100% was just going to assume it was because the tech itself was unreliable and they picked out someone who looked like him. Perhaps if I wasn't read up on the debate around this I wouldn't have thought that - but I imagine most people would assume this with confidence if they saw the headline on their feed.
  - mannykannot 5y ago
    You are making a valid, but relatively small, point about how the article is written. People who think they have learned something important from a headline without verifying what they think they have learned by applying critical analysis to the article are likely to be misinformed in all sorts of ways regardless of what they do read. Note that we are both talking about other people - we, of course, are not like that!
- deleted 5y ago
  [deleted]
elliekelly 5y ago
The AI falsely accused someone because the AI was fed garbage data. If you’re going to build an algorithm that can levy such serious charges against people it certainly seems negligent to create a process that allows random unverified training data to be input into the system. There are real people who suffer real consequences because of the programs we write. When things go wrong we don’t get to point the finger at the algorithm and the users and take no blame as the developers.
dracodoc 5y ago
The headline is accurate because that's one subject of the lawsuit. If I'm understanding it correctly after skimming through the filing(read especially page 6 and 9): 1. the plaintiff argues that in Boston case (page 6) apple store only took less than 10 minutes to identify the shoplifter, so they probably used facial recognition tech. Note the shoplifter left the store, apple found the incident, report it to police within 10 minutes, without actually detain the person. 2. I'm guessing the facial recognition is actually accurate because it was based on their previous security video, not plaintiff's real id photo(which apple probably don't have access all the time). So the facial recognition just linked one person with previous security video properly, and the id of person was based on previous record made by employee. 3. Thus there is nothing wrong here with facial recognition, but the title will definitely evoke fear and people's emotion since most people will not see the details.

threatofrain 5y ago

Not just another kid, but a friend. Ousmane Bah is an honors student, while Mamadou Barrie is potentially responsible for a string of thefts and the impersonation of his friend to the police. > "The arresting officer was able to identify the impostor as Mamadou Barrie, a friend of the Plaintiff, who apparently stole the learner’s permit from the Plaintiff," the complaint says. "These arrests specifically [noted] that Barrie had pretended to be Ousmane Bah." > The detective subsequently submitted an information request "to the NYPD’s Facial Identification Section (FIS), which identified the photograph as potentially depicting two people, one of whom was purportedly Ousmane Bah – and the other was the actual thief, Mamadou Barrie."

pishpash 5y ago
That's why you pick your "friends" carefully.
- deleted 5y ago
  [deleted]

naikrovek 5y ago

> the headline makes it sound like this was an AI falsely accusing someone based on pure randomness. no, it doesn't. the headline makes it sound like facial recognition is unreliable for some reason. the reason, in this case, is that the Apple store and/or SIS personnel gave the recognition system Ousmane's name when they weren't able to prove it was him, and the document provided for identification was not actual proof of identification. Further facial recognition matches assigned further blame to the misidentified person, increasing the evidence against him. Then, because store owners and security firms really want to put away shoplifters, they claimed that video proof of the shoplifting was routinely deleted when there was no policy describing a deletion policy within Apple or SIS, only for video proof that the individual was not Mr. Bah to later be found. in this case, facial recognition tech was unreliable entirely because of the people running it.

frogpelt 5y ago
The HN title is bad. The teen was not falsely accused by unreliable facial recognition technology. I sort of skipped through the actual headline trying to figure out how AI was involved. It was. But only after several blunders by people.
- elliekelly 5y ago
  If the teen was falsely accused and the false accusations came from the facial recognition technology then the facial recognition technology was, in fact, unreliable.
  - mattmcknight 5y ago
    The false accusations didn't come from the facial recognition technology. They came from believing the criminal using this falsely accused person's ID card which did not have a picture on it. Facial recognition tech correctly matched the face of the criminal to the other video of the criminal.
    - 411111111111111 5y ago
      Doesn't that make it worse though? Drum up false charges and they get automagically mapped to the face with nobody able to undo anything. They're now forever a criminal. Honestly, it sounds extremely dangerous in a field such as criminal investigations.
      - zo1 5y ago
        This is precisely why the "fear" or rather stigma around facial-recognition needs to go away. If it weren't there, it'd have been expected and granted that the government's job is to have valid, verified and correct biometric information on the citizen (because they're generally the ultimate/final custodian of people's identity), and the facial-recognition would have to match against that verified data, instead of some random video or fake ID. So instead we have scattered silos of partial and probably incorrect data in various places. FBI has a bit, Police department in state X has one, State Y's DMV has something wrong because X years ago this person was a victim of identity fraud, etc etc. It's a giant mess and no wonder the police and various entities in this example failed utterly. They're ultimately all victims of a system that has no single source of truth, yet we're tasking them with using "tech" to solve crime. So instead of fixing the source of truth, we blame the tech.

mannykannot 5y ago

You seem right to say it is not a case of AI falsely accusing someone based on pure randomness, but that is no reason to dismiss what happened here. The most troubling aspects are that SIS added what amounts to false information to its databases, fabricated multiple false cases against Bah, claimed to have evidence it did not have, used it to detain him and hand him over to the Paramus police for arrest (who failed in their own due diligence, partly on account of this false information), and then attempted to hide what it had done, including, apparently, destroying evidence - evidence against itself, not Bah.

handmodel 5y ago
If a DNA test identified the wrong suspect because someone mixed up the names when entering them into a database I wouldn't say that there was "unreliable DNA tech". Maybe other people didn't read the headline as I did. But if I hadn't bothered to click on it I 100% was just going to assume it was because the tech itself was unreliable and they picked out someone who looked like him. Perhaps if I wasn't read up on the debate around this I wouldn't have thought that - but I imagine most people would assume this with confidence if they saw the headline on their feed.
- mannykannot 5y ago
  You are making a valid, but relatively small, point about how the article is written. People who think they have learned something important from a headline without verifying what they think they have learned by applying critical analysis to the article are likely to be misinformed in all sorts of ways regardless of what they do read. Note that we are both talking about other people - we, of course, are not like that!
deleted 5y ago
[deleted]

elliekelly 5y ago

The AI falsely accused someone because the AI was fed garbage data. If you’re going to build an algorithm that can levy such serious charges against people it certainly seems negligent to create a process that allows random unverified training data to be input into the system. There are real people who suffer real consequences because of the programs we write. When things go wrong we don’t get to point the finger at the algorithm and the users and take no blame as the developers.

dracodoc 5y ago

The headline is accurate because that's one subject of the lawsuit. If I'm understanding it correctly after skimming through the filing(read especially page 6 and 9): 1. the plaintiff argues that in Boston case (page 6) apple store only took less than 10 minutes to identify the shoplifter, so they probably used facial recognition tech. Note the shoplifter left the store, apple found the incident, report it to police within 10 minutes, without actually detain the person. 2. I'm guessing the facial recognition is actually accurate because it was based on their previous security video, not plaintiff's real id photo(which apple probably don't have access all the time). So the facial recognition just linked one person with previous security video properly, and the id of person was based on previous record made by employee. 3. Thus there is nothing wrong here with facial recognition, but the title will definitely evoke fear and people's emotion since most people will not see the details.