3 ms·

The HN title is bad. The teen was not falsely accused by unreliable facial recognition technology. I sort of skipped through the actual headline trying to figu

The HN title is bad. The teen was not falsely accused by unreliable facial recognition technology. I sort of skipped through the actual headline trying to figure out how AI was involved. It was. But only after several blunders by people.

elliekelly 5y ago
If the teen was falsely accused and the false accusations came from the facial recognition technology then the facial recognition technology was, in fact, unreliable.
- mattmcknight 5y ago
  The false accusations didn't come from the facial recognition technology. They came from believing the criminal using this falsely accused person's ID card which did not have a picture on it. Facial recognition tech correctly matched the face of the criminal to the other video of the criminal.
  - 411111111111111 5y ago
    Doesn't that make it worse though? Drum up false charges and they get automagically mapped to the face with nobody able to undo anything. They're now forever a criminal. Honestly, it sounds extremely dangerous in a field such as criminal investigations.
    - zo1 5y ago
      This is precisely why the "fear" or rather stigma around facial-recognition needs to go away. If it weren't there, it'd have been expected and granted that the government's job is to have valid, verified and correct biometric information on the citizen (because they're generally the ultimate/final custodian of people's identity), and the facial-recognition would have to match against that verified data, instead of some random video or fake ID. So instead we have scattered silos of partial and probably incorrect data in various places. FBI has a bit, Police department in state X has one, State Y's DMV has something wrong because X years ago this person was a victim of identity fraud, etc etc. It's a giant mess and no wonder the police and various entities in this example failed utterly. They're ultimately all victims of a system that has no single source of truth, yet we're tasking them with using "tech" to solve crime. So instead of fixing the source of truth, we blame the tech.
      - A4ET8a8uTh0 5y ago
        The stigma is there, because if it was in one centralized database, the issue would be much more pronounced. It is a lot easier to destroy a company than a government.
    - mattmcknight 5y ago
      In this case, it's really the problems of names, not faces. The wrong name got attached to the charge, but the right face.
  - atq2119 5y ago
    While technically true, the use of tech is still notable here because it was used to launder the mistakes of humans. There is an incorrect tendency to ascribe intelligence to AI, which can make especially laypeople prone to assume that if an AI system makes a statement, that statement has a certain sort of validity to it that isn't necessarily justified. Garbage in garbage out applies to AI just like it applies to any other software system, and society needs to develop more resilience to this fact.
    - alisonkisk 5y ago
      AI had nothing to do with the mistake. The error was in database with poor data quality. The same would have happened if a human recognized the kid and matched him to the (stolen) id card on file.
      - mattmcknight 5y ago
        Exactly, it's just that "Teen accused of shoplifting due to unreliable data entry" doesn't get as many clicks. It's not like the database itself was unreliable, or the facial recognition was unreliable.
      - atq2119 5y ago
        You didn't engage with the core of the argument, which is that the veneer of AI is used to launder the human mistake. Increasing the distance between database input and output gets in the way of human heuristics for recognizing and fixing such mistakes. Yes, the same could have happened without AI, but the probabilities are different.
- fastball 5y ago
  The false accusations did not come from facial recognition technology.
gnud 5y ago
That's the case with all "bad AI" stories. Remember, an AI can't make a mistake until we have true AI. Until then, there's always human error - either the operator, or the programmer. In this case, it was obviously a mismatch between how the different operators treated the 'name' field - one operator put in an uncertain ID, and another one trusted the system to be correct. These stories are IMPORTANT. They might help someone think twice before either using or blindly trusting these systems.
- jjk166 5y ago
  There's still a huge difference between "people put in the correct parameters and AI nevertheless got the wrong result" and "people put in incorrect parameters and AI isn't a miracle worker that can correct it for them." You wouldn't title a story about a guy hitting his thumb with a hammer "Man attacked by unreliable hammer"

elliekelly 5y ago

If the teen was falsely accused and the false accusations came from the facial recognition technology then the facial recognition technology was, in fact, unreliable.

mattmcknight 5y ago
The false accusations didn't come from the facial recognition technology. They came from believing the criminal using this falsely accused person's ID card which did not have a picture on it. Facial recognition tech correctly matched the face of the criminal to the other video of the criminal.
- 411111111111111 5y ago
  Doesn't that make it worse though? Drum up false charges and they get automagically mapped to the face with nobody able to undo anything. They're now forever a criminal. Honestly, it sounds extremely dangerous in a field such as criminal investigations.
  - zo1 5y ago
    This is precisely why the "fear" or rather stigma around facial-recognition needs to go away. If it weren't there, it'd have been expected and granted that the government's job is to have valid, verified and correct biometric information on the citizen (because they're generally the ultimate/final custodian of people's identity), and the facial-recognition would have to match against that verified data, instead of some random video or fake ID. So instead we have scattered silos of partial and probably incorrect data in various places. FBI has a bit, Police department in state X has one, State Y's DMV has something wrong because X years ago this person was a victim of identity fraud, etc etc. It's a giant mess and no wonder the police and various entities in this example failed utterly. They're ultimately all victims of a system that has no single source of truth, yet we're tasking them with using "tech" to solve crime. So instead of fixing the source of truth, we blame the tech.
    - A4ET8a8uTh0 5y ago
      The stigma is there, because if it was in one centralized database, the issue would be much more pronounced. It is a lot easier to destroy a company than a government.
  - mattmcknight 5y ago
    In this case, it's really the problems of names, not faces. The wrong name got attached to the charge, but the right face.
- atq2119 5y ago
  While technically true, the use of tech is still notable here because it was used to launder the mistakes of humans. There is an incorrect tendency to ascribe intelligence to AI, which can make especially laypeople prone to assume that if an AI system makes a statement, that statement has a certain sort of validity to it that isn't necessarily justified. Garbage in garbage out applies to AI just like it applies to any other software system, and society needs to develop more resilience to this fact.
  - alisonkisk 5y ago
    AI had nothing to do with the mistake. The error was in database with poor data quality. The same would have happened if a human recognized the kid and matched him to the (stolen) id card on file.
    - mattmcknight 5y ago
      Exactly, it's just that "Teen accused of shoplifting due to unreliable data entry" doesn't get as many clicks. It's not like the database itself was unreliable, or the facial recognition was unreliable.
    - atq2119 5y ago
      You didn't engage with the core of the argument, which is that the veneer of AI is used to launder the human mistake. Increasing the distance between database input and output gets in the way of human heuristics for recognizing and fixing such mistakes. Yes, the same could have happened without AI, but the probabilities are different.
fastball 5y ago
The false accusations did not come from facial recognition technology.

gnud 5y ago

That's the case with all "bad AI" stories. Remember, an AI can't make a mistake until we have true AI. Until then, there's always human error - either the operator, or the programmer. In this case, it was obviously a mismatch between how the different operators treated the 'name' field - one operator put in an uncertain ID, and another one trusted the system to be correct. These stories are IMPORTANT. They might help someone think twice before either using or blindly trusting these systems.

jjk166 5y ago
There's still a huge difference between "people put in the correct parameters and AI nevertheless got the wrong result" and "people put in incorrect parameters and AI isn't a miracle worker that can correct it for them." You wouldn't title a story about a guy hitting his thumb with a hammer "Man attacked by unreliable hammer"