4 ms·

You are correct. In fact there is research that humans are not particularly good at matching people to photos... this 2014 study showed a 15% error rate (with

You are correct. In fact there is research that humans are not particularly good at matching people to photos... this 2014 study showed a 15% error rate (with photos from identification documents, high-quality standardized photos, not low-quality random angle ones!), and the real kicker is passport officers (who do this as part of their job all day and presumably care about doing it well) surprisingly performed no better than random undergrads... https://qz.com/251791/passport-officers-are-no-better-at-performing-photo-id-checks-than-you-are https://qz.com/251791/passport-officers-are-no-better-at-per... (Someone is now going to suggest, right, that's why we should have computers do it instead, bringing the circle back around again... but just no).

crazygringo 3y ago
Which is precisely why you combine the two systems. If humans get it wrong 15% of the time, that means they're reducing false positives by 85%. That's the whole point of combining facial recognition with human review. You still have to decide what to do with that final information considering that it's still not perfect. But you should never be relying on facial recognition by itself. Even if humans are imperfect, they still improve the accuracy, and can make the final call "nope I'm just not sure, I'm not going to take action".
- ProjectArcturis 3y ago
  The question is how correlated are human and machine errors. I would guess fairly strongly, in which case human review would add little additional accuracy.
  - crazygringo 3y ago
    > I would guess fairly strongly No, evidence suggests they are not. We're all very aware of the horror stories of AI misidentifying species. Different AI systems are different, but in general AI seems to make quite different classes of mistakes from humans. Our brains work very differently from current statistical models. So human review absolutely adds accuracy, you can generally assume. And it adds human-accountable judgment which is just as important.
    - ProjectArcturis 3y ago
      Ultimately it's an empirical question, and unless someone has published the research, you and I will not be able to know for sure. I think there will be a lot of cases where two people simply look a lot alike, and any observer would have difficulty distinguishing them.
- sokoloff 3y ago
  The error reduction from applying a different system applies if the source of the error does not manifest itself in both systems (if errors are uncorrelated, you can get an improvement that looks like that implied by the simple application of statistics, but not if they are correlated). As an example, you can combine human review with automatic facial detection of identical twins and likely not see much (any?) reduction in error rate at all. Two independent "85% accurate" humans are not 97.7% accurate for identical twins either.
gruez 3y ago
>In fact there is research that humans are not particularly good at matching people to photos... >(Someone is now going to suggest, right, that's why we should have computers do it instead, bringing the circle back around again... but just no). So where does this leave us for facial recognition? Should we ban both computer and human facial recognition, because they're both flawed? How would that be enforced? If a store employee thinks they recognize someone from a few minutes ago, are they supposed to ignore that fact, and pretend that they're different people, on the off chance that the guy might be someone else[1]? [1] https://www.youtube.com/watch?v=hGsJ3reXz-k https://www.youtube.com/watch?v=hGsJ3reXz-k
- jerf 3y ago
  "So where does this leave us for facial recognition?" Where it leaves us is that it doesn't work, and it can't work. I see no evidence that there is some big reservoir of facial recognition quality to be extracted from the same basic data set. There is all sorts of reasons to believe that it is simply impossible to create a system that can be given a small percentage of the population as the targets and pick them out from millions of samples correctly. Of all the disciplines, those trained in computer science should be aware of the concept that problems can be fundamentally hard or unsolvable. However, I've been careful to phrase what I think may be fundamentally unsolvable as being related to "the same basic data set". Expansion of the data set provides other possibilities, and while I'm not ready to declare that adding that data will certainly solve the problem, I'm not ready to declare it as fundamentally unsolvable either. Add portable device tracking, gait analysis, speech analysis, anything else some clever clog can think of, and probably drop the requirement that de facto minimum wagers be asked to confront nominal criminals (I would assert there is no solution to the mismatched incentives there), and the problem may well be solvable. It would, however, require Rite Aid and anyone else planning to use this sort of thing to radically upgrade their hardware.
  - gruez 3y ago
    >Where it leaves us is that it doesn't work, and it can't work. You didn't answer the second part of my comment: "How would that be enforced? If a store employee thinks they recognize someone from a few minutes ago, are they supposed to ignore that fact, and pretend that they're different people, on the off chance that the guy might be someone else[1]?" >and probably drop the requirement that de facto minimum wagers be asked to confront nominal criminals Are you saying this on the basis that they're not qualified to make an identification, or that confrontation would put them at risk of violence? If it's the latter it really doesn't have anything to do with facial recognition. It would still apply even if replaced facial recognition with a 100% accurate oracle.
seanmcdirmid 3y ago
The computers do it for passports photos at self check kiosks in immigration. The thinking is that the computers are better at it anyways, so no fidelity is lost vs a manned checkpoint. A false negative is easy enough to deal with anyways, and that false positive could have happened at the manned checkpoint as well. Banned lists are simply impossible to implement in general. Instead, police should be more active in shoplifting and organized theft cases. Someone shoplifts, it’s at least a felony, get them in the system, and apply penalties to repeat offenders. Having stores sort this out themselves is just crazy.

crazygringo 3y ago

Which is precisely why you combine the two systems. If humans get it wrong 15% of the time, that means they're reducing false positives by 85%. That's the whole point of combining facial recognition with human review. You still have to decide what to do with that final information considering that it's still not perfect. But you should never be relying on facial recognition by itself. Even if humans are imperfect, they still improve the accuracy, and can make the final call "nope I'm just not sure, I'm not going to take action".

ProjectArcturis 3y ago
The question is how correlated are human and machine errors. I would guess fairly strongly, in which case human review would add little additional accuracy.
- crazygringo 3y ago
  > I would guess fairly strongly No, evidence suggests they are not. We're all very aware of the horror stories of AI misidentifying species. Different AI systems are different, but in general AI seems to make quite different classes of mistakes from humans. Our brains work very differently from current statistical models. So human review absolutely adds accuracy, you can generally assume. And it adds human-accountable judgment which is just as important.
  - ProjectArcturis 3y ago
    Ultimately it's an empirical question, and unless someone has published the research, you and I will not be able to know for sure. I think there will be a lot of cases where two people simply look a lot alike, and any observer would have difficulty distinguishing them.
sokoloff 3y ago
The error reduction from applying a different system applies if the source of the error does not manifest itself in both systems (if errors are uncorrelated, you can get an improvement that looks like that implied by the simple application of statistics, but not if they are correlated). As an example, you can combine human review with automatic facial detection of identical twins and likely not see much (any?) reduction in error rate at all. Two independent "85% accurate" humans are not 97.7% accurate for identical twins either.

gruez 3y ago

>In fact there is research that humans are not particularly good at matching people to photos... >(Someone is now going to suggest, right, that's why we should have computers do it instead, bringing the circle back around again... but just no). So where does this leave us for facial recognition? Should we ban both computer and human facial recognition, because they're both flawed? How would that be enforced? If a store employee thinks they recognize someone from a few minutes ago, are they supposed to ignore that fact, and pretend that they're different people, on the off chance that the guy might be someone else[1]? [1] https://www.youtube.com/watch?v=hGsJ3reXz-k https://www.youtube.com/watch?v=hGsJ3reXz-k

jerf 3y ago
"So where does this leave us for facial recognition?" Where it leaves us is that it doesn't work, and it can't work. I see no evidence that there is some big reservoir of facial recognition quality to be extracted from the same basic data set. There is all sorts of reasons to believe that it is simply impossible to create a system that can be given a small percentage of the population as the targets and pick them out from millions of samples correctly. Of all the disciplines, those trained in computer science should be aware of the concept that problems can be fundamentally hard or unsolvable. However, I've been careful to phrase what I think may be fundamentally unsolvable as being related to "the same basic data set". Expansion of the data set provides other possibilities, and while I'm not ready to declare that adding that data will certainly solve the problem, I'm not ready to declare it as fundamentally unsolvable either. Add portable device tracking, gait analysis, speech analysis, anything else some clever clog can think of, and probably drop the requirement that de facto minimum wagers be asked to confront nominal criminals (I would assert there is no solution to the mismatched incentives there), and the problem may well be solvable. It would, however, require Rite Aid and anyone else planning to use this sort of thing to radically upgrade their hardware.
- gruez 3y ago
  >Where it leaves us is that it doesn't work, and it can't work. You didn't answer the second part of my comment: "How would that be enforced? If a store employee thinks they recognize someone from a few minutes ago, are they supposed to ignore that fact, and pretend that they're different people, on the off chance that the guy might be someone else[1]?" >and probably drop the requirement that de facto minimum wagers be asked to confront nominal criminals Are you saying this on the basis that they're not qualified to make an identification, or that confrontation would put them at risk of violence? If it's the latter it really doesn't have anything to do with facial recognition. It would still apply even if replaced facial recognition with a 100% accurate oracle.

seanmcdirmid 3y ago

The computers do it for passports photos at self check kiosks in immigration. The thinking is that the computers are better at it anyways, so no fidelity is lost vs a manned checkpoint. A false negative is easy enough to deal with anyways, and that false positive could have happened at the manned checkpoint as well. Banned lists are simply impossible to implement in general. Instead, police should be more active in shoplifting and organized theft cases. Someone shoplifts, it’s at least a felony, get them in the system, and apply penalties to repeat offenders. Having stores sort this out themselves is just crazy.