7 ms·

The article seems to hang on the false positives being higher for certain minorities, but makes no mention of the false negatives being higher as well. So whil

by daenz 7y ago

The article seems to hang on the false positives being higher for certain minorities, but makes no mention of the false negatives being higher as well. So while some ethnic minorities might be more likely to match incorrectly to a known person of interest, they're also more likely to be let through if they are indeed that person. I think that the first case is definitely more damaging on the whole, but I still find it misleading to not mention the specific scope of the "racial bias."

Barrin92 7y ago
it doesn't seem relevant. In a country like the US the law pursues criminal individuals, not criminal groups. The fact that the system also falsely negatively identified people doesn't somehow 'balance out' false positives, it's not like the point of these systems is to randomly catch 100 African-Americans. (although given the political ideologies of some of the people in the surveillance tech space maybe it is) The fact that it produces errors in both directions just makes the system even worse in total.
- shadowgovt 7y ago
  Exactly. If anything, it means the problem is worse than the article indicates.
- daenz 7y ago
  >The fact that it produces errors in both directions just makes the system even worse in total. It's almost like the purpose of the article isn't to show the tech is overall empirically bad.
- allovernow 7y ago
  >The fact that the system also falsely negatively identified people doesn't somehow 'balance out' false positives, it's not like the point of these systems is to randomly catch 100 African-Americans. If they're disproportionatly committing crimes, skin color becomes a strong prior. It would be rational, iff the goal is to lower crime rates. But I don't know that it is morally OK. Also I'm not sure if over representation is the source of bias in this particular case, haven't read the article yet. Edit: yeah, this article is just a smear, wapo is salivating over anything they think confirms that police are racially biased. It doesn't mention anything technical about the training data or typical nets being used. I'm not advocating for facial recognition tech, I'm personally against it, but racial bias coming from objective data is the least of your potential worries.
- AnthonyMouse 7y ago
  > it doesn't seem relevant. The reason it's relevant is that false positives are always a trade off against false negatives, so the "easiest" way to reduce false positives is to increase false negatives. But false negatives are also very bad, so we can't really do that or we're just trading one problem for another instead of actually solving anything. The only real solution is to improve the overall accuracy of the system, but that's easier said than done. Some of the main sources of the inaccuracy are intrinsic. The population in question is a smaller proportion of the general population so there is less training data available. People with darker skin absorb rather than reflect light, which makes it harder for the system to identify them. So the problem isn't some racist schmuck who just needs to be fired and the system will get more accurate, it's a consequence of demographics and physics. There may be a solution somebody can find, but there isn't guaranteed to be, and in the meantime there's not a whole lot you can do other than trying to find one. Or maybe discontinuing the system entirely.
  - lopmotr 7y ago
    Increasing overall accuracy won't help if it's applied to all ethnicities. It has to be an increase only for the affected type of people. But if you're going to do that, you could also just artificially cripple the accuracy for the too-accurate groups. That sounds like the easiest solution, if it's actually a problem to be solved in the first place.
    - SamReidHughes 7y ago
      No, it would help a lot. Let's say you shrink mistakes from 10% and 20% to 1% and 2%. That means the gap in misidentification likelihood has shrunk from 10% to 1%.
      - lopmotr 7y ago
        Huh? Isn't that just clever accounting? 2% is still twice as much as 1%. If a black person and a white person each travel the same number of times, the black can still expect twice as many additional checks as the white, the same as before the improvement.
- dahfizz 7y ago
  The difference between bias and inaccuracy is extremely important, especially legally.
egdod 7y ago
Yeah, this is not a racial bias. It’s race-based inaccuracy.
- smadge 7y ago
  I think that fulfills the definition of both statistical and legal bias.
  - skj 7y ago
    Bias and inaccuracy are two different ways to get things wrong. Briefly, bias is the wrong mean, and inaccuracy is high variance.
    - smadge 7y ago
      Thanks! That clarifies the distinction.
    - m0zg 7y ago
      And what people also don't realize is machine learning system will get massively more accurate over time, and its accuracy landscape will generally follow the distribution of the training set. Academic training sets this is typically bootstrapped on are very different from what you'd encounter in the US criminal justice system, but as the system is fed real data it can get crazy good at what it does. Superhuman accuracy facial recognition systems have existed for a while now. Humans just aren't that good at matching faces, and it's not like humans are bias-free either.
    - learnstats2 7y ago
      There is an entirely different mean number of false positive stops. So, by your statistical definition, there's a bias.

by daenz 7y ago

Barrin92 7y ago

it doesn't seem relevant. In a country like the US the law pursues criminal individuals, not criminal groups. The fact that the system also falsely negatively identified people doesn't somehow 'balance out' false positives, it's not like the point of these systems is to randomly catch 100 African-Americans. (although given the political ideologies of some of the people in the surveillance tech space maybe it is) The fact that it produces errors in both directions just makes the system even worse in total.

shadowgovt 7y ago
Exactly. If anything, it means the problem is worse than the article indicates.
daenz 7y ago
>The fact that it produces errors in both directions just makes the system even worse in total. It's almost like the purpose of the article isn't to show the tech is overall empirically bad.
allovernow 7y ago
>The fact that the system also falsely negatively identified people doesn't somehow 'balance out' false positives, it's not like the point of these systems is to randomly catch 100 African-Americans. If they're disproportionatly committing crimes, skin color becomes a strong prior. It would be rational, iff the goal is to lower crime rates. But I don't know that it is morally OK. Also I'm not sure if over representation is the source of bias in this particular case, haven't read the article yet. Edit: yeah, this article is just a smear, wapo is salivating over anything they think confirms that police are racially biased. It doesn't mention anything technical about the training data or typical nets being used. I'm not advocating for facial recognition tech, I'm personally against it, but racial bias coming from objective data is the least of your potential worries.
AnthonyMouse 7y ago
> it doesn't seem relevant. The reason it's relevant is that false positives are always a trade off against false negatives, so the "easiest" way to reduce false positives is to increase false negatives. But false negatives are also very bad, so we can't really do that or we're just trading one problem for another instead of actually solving anything. The only real solution is to improve the overall accuracy of the system, but that's easier said than done. Some of the main sources of the inaccuracy are intrinsic. The population in question is a smaller proportion of the general population so there is less training data available. People with darker skin absorb rather than reflect light, which makes it harder for the system to identify them. So the problem isn't some racist schmuck who just needs to be fired and the system will get more accurate, it's a consequence of demographics and physics. There may be a solution somebody can find, but there isn't guaranteed to be, and in the meantime there's not a whole lot you can do other than trying to find one. Or maybe discontinuing the system entirely.
- lopmotr 7y ago
  Increasing overall accuracy won't help if it's applied to all ethnicities. It has to be an increase only for the affected type of people. But if you're going to do that, you could also just artificially cripple the accuracy for the too-accurate groups. That sounds like the easiest solution, if it's actually a problem to be solved in the first place.
  - SamReidHughes 7y ago
    No, it would help a lot. Let's say you shrink mistakes from 10% and 20% to 1% and 2%. That means the gap in misidentification likelihood has shrunk from 10% to 1%.
    - lopmotr 7y ago
      Huh? Isn't that just clever accounting? 2% is still twice as much as 1%. If a black person and a white person each travel the same number of times, the black can still expect twice as many additional checks as the white, the same as before the improvement.
dahfizz 7y ago
The difference between bias and inaccuracy is extremely important, especially legally.

egdod 7y ago

Yeah, this is not a racial bias. It’s race-based inaccuracy.

smadge 7y ago
I think that fulfills the definition of both statistical and legal bias.
- skj 7y ago
  Bias and inaccuracy are two different ways to get things wrong. Briefly, bias is the wrong mean, and inaccuracy is high variance.
  - smadge 7y ago
    Thanks! That clarifies the distinction.
  - m0zg 7y ago
    And what people also don't realize is machine learning system will get massively more accurate over time, and its accuracy landscape will generally follow the distribution of the training set. Academic training sets this is typically bootstrapped on are very different from what you'd encounter in the US criminal justice system, but as the system is fed real data it can get crazy good at what it does. Superhuman accuracy facial recognition systems have existed for a while now. Humans just aren't that good at matching faces, and it's not like humans are bias-free either.
  - learnstats2 7y ago
    There is an entirely different mean number of false positive stops. So, by your statistical definition, there's a bias.