9 ms·

Facial recognition tech wrongly identified 2,000 people as potential criminals

53 points by marchenko 8y ago

setquk 8y ago
Sounds like someone sold them a promise and no one wants to be accountable for it underperforming.
talltimtom 8y ago
The algorithm isn’t arresting people. It’s cutting down the suspect pool from 17000 to 2500, the fact that 2000 of those are innoscent while 500 are criminals isn’t a damning issue, it’s just a point where it could improve. Without the system the 500 people would not have been identified at all.
- tzmudzin 8y ago
  > 2,470 potential matches were identified. [...] > 2,297 - 92% - were found to be "false positives". ... so we have 173 true positives, but 450 people were arrested. I guess the 277 wrongfully arrested ones may have an opinion different from yours.
  - mic47 8y ago
    That assumes that both numbers are for the same period and that each arrest correspond to 1 match, which most likely will not be be true. For example if system identified only subset of people, but when you make arrest you arrest all. That means that now you have more arrest attributed to system, than number of matches. Since article does not do any attempt to answer these discrepancies, there is high chance the devil is in the details of definitions of those numbers.
  - Silhouette 8y ago
    You appear to be comparing two unrelated data points. The 92% and related figures were for one specific occasion. The 450 was total arrests since the technology was introduced. The article specifically reports the police saying no-one has been arrested as a result of an incorrect match.
- Molaxx 8y ago
  Why do you consider 170000 law abiding citizens as potential suspects? That's an extremely paranoid approach to law enforcement, especially when according to basic criminal law, these citizens are supposed to be considered innocent by default. Mass automatic surveillance of this reverses the idea of innocent until proven guilty without actually aknowleding it. That's scary
eddz 8y ago
As long as the data is used as one of many guideline sources which is scrutinised by human officers before the individuals are suspected and acted upon, I personally don't see a problem with the inaccuracy (which will get better over time).
- mtgx 8y ago
  > As long as the data is used as one of many guideline sources which is scrutinised by human officers What makes you think that's going to happen? If anything, the trend is in the opposite direction - reducing the need for humans within automated systems as much as possible. If a fast food store manager brings in robots to serve people food, do you think he'll still want to hire people around to make sure the robots are serving the right food to people? No, if he brought in robots, then it's because he wants to get rid of as many human employees as possible. Maybe the humans will be there for like the first month or two while the robots are being tested, but that's it - or at least that would be the goal. Look how fast Musk wanted to get rid of human employees at Tesla, too. Complete 100% automation will be the goal for both companies and governments. Even the Pentagon is rushing to allow its AI drones to kill "targets" on their own (thanks, Google). You don't think governments will rush to get humans out of the equation for more "trivial" stuff like arresting people? If 20% of the people are wrongfully accused, but the system catches 99.9% of the actual criminals, then they will be very happy with that system. Just like Facebook is happy to censor 10%, 20%, 40% other innocent content, as long as it can brag to Congress that it "censored 99% of the terrorist content" (which probably isn't true, either). They don't care about the collateral damage.
  - deleted 8y ago
    [deleted]
  - eddz 8y ago
    Some programs are designed to replace humans, others are designed as tools. Automation does not always equal the replacement of humans. I appreciate your analogies but there is not evidence in the article to suggests that police and investigators are being replaced by automated systems. Data such as this can be used to eliminate a lot of the investigative work which usually goes into law enforcement. Are you familiar with a police lineup? Police detain a selection of suspects based on various characteristics and ask witnesses to select the perpetrator, who may or may not be present at the lineup. Neither a police lineup nor facial recognition are accurate or reliable when used alone, but when used in combination with other facts, tools, and resources an arrest can be made with increased speed and confidence.
akakjs 8y ago
While the article does go into the detail of the accuracy of the deployed systems, it's both sad and worrying that the counter arguments to mass facial recognition are reduced to a single positioning quote.
mic47 8y ago
[I focus only on the performance of facial recognition system, not on whether it's good idea to have such systems in place.] From the systems performance point of view, this is non-story. If you deploy detection system, you have to make trade-off between precision (how many false positives you will have) and recall (how many false negatives -- misses) you will have [1]. How you do this trade-off depends on cost of false positive, and cost of false negative. For example, if false positive means, that someone will point camera to that part of stadium and someone will manually check if it's correct [2], and then arrest, then 2000 false positives might be good performance if recall is good, because there is no negative impact on false positive, only some wasted effort by police. On the other hand, if this AI match would be used as evidence in court, then the it would be terrible, as it would cause much more harm than good. From the article it seems that police is using it more closely to the first case, than to the second case, so the performance of this system might be good, so really the article is non-story. [1] I simplified the "precision" and "recall". [2] Or send more security guards / police to that area, or whatever discrete measure.
- Retric 8y ago
  People being arrested clearly qualifies as a negative impact. Asking for ID and arresting even 1 of those 2000 people also qualifys as a negative. Worse even if police simply go up to these people to do a manual comparison that's a downside as police confrontations risk death.
  - mic47 8y ago
    Article does not claim happening any one of those, as there is no information about how police handle this. [edit] actually they claim that no false positive was arrested. But not others.
    - dpwm 8y ago
      Article in fact claims police force said no-one was arrested after an incorrect match. > It also said no-one had been arrested after an incorrect match.
  - 8y ago
pressurefree 8y ago
let me guess... its 3,000 potential criminals...

53 points by marchenko 8y ago

Sounds like someone sold them a promise and no one wants to be accountable for it underperforming.

The algorithm isn’t arresting people. It’s cutting down the suspect pool from 17000 to 2500, the fact that 2000 of those are innoscent while 500 are criminals isn’t a damning issue, it’s just a point where it could improve. Without the system the 500 people would not have been identified at all.

tzmudzin 8y ago
> 2,470 potential matches were identified. [...] > 2,297 - 92% - were found to be "false positives". ... so we have 173 true positives, but 450 people were arrested. I guess the 277 wrongfully arrested ones may have an opinion different from yours.
- mic47 8y ago
  That assumes that both numbers are for the same period and that each arrest correspond to 1 match, which most likely will not be be true. For example if system identified only subset of people, but when you make arrest you arrest all. That means that now you have more arrest attributed to system, than number of matches. Since article does not do any attempt to answer these discrepancies, there is high chance the devil is in the details of definitions of those numbers.
- Silhouette 8y ago
  You appear to be comparing two unrelated data points. The 92% and related figures were for one specific occasion. The 450 was total arrests since the technology was introduced. The article specifically reports the police saying no-one has been arrested as a result of an incorrect match.
Molaxx 8y ago
Why do you consider 170000 law abiding citizens as potential suspects? That's an extremely paranoid approach to law enforcement, especially when according to basic criminal law, these citizens are supposed to be considered innocent by default. Mass automatic surveillance of this reverses the idea of innocent until proven guilty without actually aknowleding it. That's scary

eddz 8y ago

As long as the data is used as one of many guideline sources which is scrutinised by human officers before the individuals are suspected and acted upon, I personally don't see a problem with the inaccuracy (which will get better over time).

mtgx 8y ago
> As long as the data is used as one of many guideline sources which is scrutinised by human officers What makes you think that's going to happen? If anything, the trend is in the opposite direction - reducing the need for humans within automated systems as much as possible. If a fast food store manager brings in robots to serve people food, do you think he'll still want to hire people around to make sure the robots are serving the right food to people? No, if he brought in robots, then it's because he wants to get rid of as many human employees as possible. Maybe the humans will be there for like the first month or two while the robots are being tested, but that's it - or at least that would be the goal. Look how fast Musk wanted to get rid of human employees at Tesla, too. Complete 100% automation will be the goal for both companies and governments. Even the Pentagon is rushing to allow its AI drones to kill "targets" on their own (thanks, Google). You don't think governments will rush to get humans out of the equation for more "trivial" stuff like arresting people? If 20% of the people are wrongfully accused, but the system catches 99.9% of the actual criminals, then they will be very happy with that system. Just like Facebook is happy to censor 10%, 20%, 40% other innocent content, as long as it can brag to Congress that it "censored 99% of the terrorist content" (which probably isn't true, either). They don't care about the collateral damage.
- deleted 8y ago
  [deleted]
- eddz 8y ago
  Some programs are designed to replace humans, others are designed as tools. Automation does not always equal the replacement of humans. I appreciate your analogies but there is not evidence in the article to suggests that police and investigators are being replaced by automated systems. Data such as this can be used to eliminate a lot of the investigative work which usually goes into law enforcement. Are you familiar with a police lineup? Police detain a selection of suspects based on various characteristics and ask witnesses to select the perpetrator, who may or may not be present at the lineup. Neither a police lineup nor facial recognition are accurate or reliable when used alone, but when used in combination with other facts, tools, and resources an arrest can be made with increased speed and confidence.

akakjs 8y ago

While the article does go into the detail of the accuracy of the deployed systems, it's both sad and worrying that the counter arguments to mass facial recognition are reduced to a single positioning quote.

mic47 8y ago

[I focus only on the performance of facial recognition system, not on whether it's good idea to have such systems in place.] From the systems performance point of view, this is non-story. If you deploy detection system, you have to make trade-off between precision (how many false positives you will have) and recall (how many false negatives -- misses) you will have [1]. How you do this trade-off depends on cost of false positive, and cost of false negative. For example, if false positive means, that someone will point camera to that part of stadium and someone will manually check if it's correct [2], and then arrest, then 2000 false positives might be good performance if recall is good, because there is no negative impact on false positive, only some wasted effort by police. On the other hand, if this AI match would be used as evidence in court, then the it would be terrible, as it would cause much more harm than good. From the article it seems that police is using it more closely to the first case, than to the second case, so the performance of this system might be good, so really the article is non-story. [1] I simplified the "precision" and "recall". [2] Or send more security guards / police to that area, or whatever discrete measure.

Retric 8y ago
People being arrested clearly qualifies as a negative impact. Asking for ID and arresting even 1 of those 2000 people also qualifys as a negative. Worse even if police simply go up to these people to do a manual comparison that's a downside as police confrontations risk death.
- mic47 8y ago
  Article does not claim happening any one of those, as there is no information about how police handle this. [edit] actually they claim that no false positive was arrested. But not others.
  - dpwm 8y ago
    Article in fact claims police force said no-one was arrested after an incorrect match. > It also said no-one had been arrested after an incorrect match.
- 8y ago

pressurefree 8y ago

let me guess... its 3,000 potential criminals...