4 ms·

> the headline makes it sound like this was an AI falsely accusing someone based on pure randomness. no, it doesn't. the headline makes it sound like facial r

> the headline makes it sound like this was an AI falsely accusing someone based on pure randomness. no, it doesn't. the headline makes it sound like facial recognition is unreliable for some reason. the reason, in this case, is that the Apple store and/or SIS personnel gave the recognition system Ousmane's name when they weren't able to prove it was him, and the document provided for identification was not actual proof of identification. Further facial recognition matches assigned further blame to the misidentified person, increasing the evidence against him. Then, because store owners and security firms really want to put away shoplifters, they claimed that video proof of the shoplifting was routinely deleted when there was no policy describing a deletion policy within Apple or SIS, only for video proof that the individual was not Mr. Bah to later be found. in this case, facial recognition tech was unreliable entirely because of the people running it.

frogpelt 5y ago
The HN title is bad. The teen was not falsely accused by unreliable facial recognition technology. I sort of skipped through the actual headline trying to figure out how AI was involved. It was. But only after several blunders by people.
- elliekelly 5y ago
  If the teen was falsely accused and the false accusations came from the facial recognition technology then the facial recognition technology was, in fact, unreliable.
  - mattmcknight 5y ago
    The false accusations didn't come from the facial recognition technology. They came from believing the criminal using this falsely accused person's ID card which did not have a picture on it. Facial recognition tech correctly matched the face of the criminal to the other video of the criminal.
    - 411111111111111 5y ago
      Doesn't that make it worse though? Drum up false charges and they get automagically mapped to the face with nobody able to undo anything. They're now forever a criminal. Honestly, it sounds extremely dangerous in a field such as criminal investigations.
      - zo1 5y ago
        This is precisely why the "fear" or rather stigma around facial-recognition needs to go away. If it weren't there, it'd have been expected and granted that the government's job is to have valid, verified and correct biometric information on the citizen (because they're generally the ultimate/final custodian of people's identity), and the facial-recognition would have to match against that verified data, instead of some random video or fake ID. So instead we have scattered silos of partial and probably incorrect data in various places. FBI has a bit, Police department in state X has one, State Y's DMV has something wrong because X years ago this person was a victim of identity fraud, etc etc. It's a giant mess and no wonder the police and various entities in this example failed utterly. They're ultimately all victims of a system that has no single source of truth, yet we're tasking them with using "tech" to solve crime. So instead of fixing the source of truth, we blame the tech.
        A4ET8a8uTh0 5y ago
        The stigma is there, because if it was in one centralized database, the issue would be much more pronounced. It is a lot easier to destroy a company than a government.
      - mattmcknight 5y ago
        In this case, it's really the problems of names, not faces. The wrong name got attached to the charge, but the right face.
    - atq2119 5y ago
      While technically true, the use of tech is still notable here because it was used to launder the mistakes of humans. There is an incorrect tendency to ascribe intelligence to AI, which can make especially laypeople prone to assume that if an AI system makes a statement, that statement has a certain sort of validity to it that isn't necessarily justified. Garbage in garbage out applies to AI just like it applies to any other software system, and society needs to develop more resilience to this fact.
  - fastball 5y ago
    The false accusations did not come from facial recognition technology.
- gnud 5y ago
  That's the case with all "bad AI" stories. Remember, an AI can't make a mistake until we have true AI. Until then, there's always human error - either the operator, or the programmer. In this case, it was obviously a mismatch between how the different operators treated the 'name' field - one operator put in an uncertain ID, and another one trusted the system to be correct. These stories are IMPORTANT. They might help someone think twice before either using or blindly trusting these systems.
  - jjk166 5y ago
    There's still a huge difference between "people put in the correct parameters and AI nevertheless got the wrong result" and "people put in incorrect parameters and AI isn't a miracle worker that can correct it for them." You wouldn't title a story about a guy hitting his thumb with a hammer "Man attacked by unreliable hammer"
wyqydsyq 5y ago
> in this case, facial recognition tech was unreliable entirely because of the people running it. Wrong. In this case the facial recognition tech itself was arguably not unreliable. The human operators were the unreliable factor. The recognition tech reliably tagged the impersonator as Ousmane exactly as it was instructed to do. The system worked exactly as intended. It is the human operator whose intention was wrong. This has nothing to do with AI being unreliable and everything to do with the employees of this SIS company going "yep kid's black, he's the one who did it" without half a thought.
- harry8 5y ago
  >This has nothing to do with AI being unreliable This has everything to do with AI being unreliable. AI is unreliable. It was unreliable yet again here. Anyone relying on AI as the evidence in a prosecution needs to have their faces rubbed in the ignorance. AI is as unreliable as any human being. In fact, it is more unreliable because people understand the limits of the reliability of humans and look for corroboration and examine the evidence with a mind on its potential shortcomings. Yet there is still this pervasive and pernicious belief that "Computers don't lie" which is completely and utterly false and worth repeating often, with emphasis until it is part of human collective understanding of the world to the same degree that water is wet. "Computer says x" by itself counts for nothing. Anyone presenting evidence like that by alone should be presenting a huge red flag that something nasty is going on, via either malice or sheer incompetence.
  - jjk166 5y ago
    The AI was reliable. It correctly identified that the individuals in two videos were the same person. The problem was that one human recorded a name that they should have known was not real, and another person read that name in a report and arrested someone with that name. The story would have been no different if a human had correctly recognized the same person in both videos.
- sb8244 5y ago
  IMO, the operators of the AI are part of the production execution of that AI. There were not proper constraints in place to prevent this from happening, and it wasn't fixed after the first time(s) that it occurred. Sure you can argue that the pure tech worked as designed, but the system includes the human element as much as the tech element.
deleted 5y ago
[deleted]

frogpelt 5y ago

The HN title is bad. The teen was not falsely accused by unreliable facial recognition technology. I sort of skipped through the actual headline trying to figure out how AI was involved. It was. But only after several blunders by people.

elliekelly 5y ago
If the teen was falsely accused and the false accusations came from the facial recognition technology then the facial recognition technology was, in fact, unreliable.
- mattmcknight 5y ago
  The false accusations didn't come from the facial recognition technology. They came from believing the criminal using this falsely accused person's ID card which did not have a picture on it. Facial recognition tech correctly matched the face of the criminal to the other video of the criminal.
  - 411111111111111 5y ago
    Doesn't that make it worse though? Drum up false charges and they get automagically mapped to the face with nobody able to undo anything. They're now forever a criminal. Honestly, it sounds extremely dangerous in a field such as criminal investigations.
    - zo1 5y ago
      This is precisely why the "fear" or rather stigma around facial-recognition needs to go away. If it weren't there, it'd have been expected and granted that the government's job is to have valid, verified and correct biometric information on the citizen (because they're generally the ultimate/final custodian of people's identity), and the facial-recognition would have to match against that verified data, instead of some random video or fake ID. So instead we have scattered silos of partial and probably incorrect data in various places. FBI has a bit, Police department in state X has one, State Y's DMV has something wrong because X years ago this person was a victim of identity fraud, etc etc. It's a giant mess and no wonder the police and various entities in this example failed utterly. They're ultimately all victims of a system that has no single source of truth, yet we're tasking them with using "tech" to solve crime. So instead of fixing the source of truth, we blame the tech.
      - A4ET8a8uTh0 5y ago
        The stigma is there, because if it was in one centralized database, the issue would be much more pronounced. It is a lot easier to destroy a company than a government.
    - mattmcknight 5y ago
      In this case, it's really the problems of names, not faces. The wrong name got attached to the charge, but the right face.
  - atq2119 5y ago
    While technically true, the use of tech is still notable here because it was used to launder the mistakes of humans. There is an incorrect tendency to ascribe intelligence to AI, which can make especially laypeople prone to assume that if an AI system makes a statement, that statement has a certain sort of validity to it that isn't necessarily justified. Garbage in garbage out applies to AI just like it applies to any other software system, and society needs to develop more resilience to this fact.
- fastball 5y ago
  The false accusations did not come from facial recognition technology.
gnud 5y ago
That's the case with all "bad AI" stories. Remember, an AI can't make a mistake until we have true AI. Until then, there's always human error - either the operator, or the programmer. In this case, it was obviously a mismatch between how the different operators treated the 'name' field - one operator put in an uncertain ID, and another one trusted the system to be correct. These stories are IMPORTANT. They might help someone think twice before either using or blindly trusting these systems.
- jjk166 5y ago
  There's still a huge difference between "people put in the correct parameters and AI nevertheless got the wrong result" and "people put in incorrect parameters and AI isn't a miracle worker that can correct it for them." You wouldn't title a story about a guy hitting his thumb with a hammer "Man attacked by unreliable hammer"

wyqydsyq 5y ago

> in this case, facial recognition tech was unreliable entirely because of the people running it. Wrong. In this case the facial recognition tech itself was arguably not unreliable. The human operators were the unreliable factor. The recognition tech reliably tagged the impersonator as Ousmane exactly as it was instructed to do. The system worked exactly as intended. It is the human operator whose intention was wrong. This has nothing to do with AI being unreliable and everything to do with the employees of this SIS company going "yep kid's black, he's the one who did it" without half a thought.

harry8 5y ago
>This has nothing to do with AI being unreliable This has everything to do with AI being unreliable. AI is unreliable. It was unreliable yet again here. Anyone relying on AI as the evidence in a prosecution needs to have their faces rubbed in the ignorance. AI is as unreliable as any human being. In fact, it is more unreliable because people understand the limits of the reliability of humans and look for corroboration and examine the evidence with a mind on its potential shortcomings. Yet there is still this pervasive and pernicious belief that "Computers don't lie" which is completely and utterly false and worth repeating often, with emphasis until it is part of human collective understanding of the world to the same degree that water is wet. "Computer says x" by itself counts for nothing. Anyone presenting evidence like that by alone should be presenting a huge red flag that something nasty is going on, via either malice or sheer incompetence.
- jjk166 5y ago
  The AI was reliable. It correctly identified that the individuals in two videos were the same person. The problem was that one human recorded a name that they should have known was not real, and another person read that name in a report and arrested someone with that name. The story would have been no different if a human had correctly recognized the same person in both videos.
sb8244 5y ago
IMO, the operators of the AI are part of the production execution of that AI. There were not proper constraints in place to prevent this from happening, and it wasn't fixed after the first time(s) that it occurred. Sure you can argue that the pure tech worked as designed, but the system includes the human element as much as the tech element.

deleted 5y ago

[deleted]