5 ms·

This is short but interesting. Microsoft is aware of the downsides of false positives. In this case California police officers would use facial recognition on

• samayylmao 8y ago
  Seems like a situation where doing the right thing is luckily in the best interest of Microsoft.
  • matt4077 8y ago
    That’s doesn’t square with Amazon’s decision to do the opposite, except in that this “best for the company” argument is so mendable it can be used to argue for anything and its opposite. A company can make moral decisions, and the fiction that lawsuits are likely or even plausible is almost entirely mythical.
• SilasX 8y ago
  Wait, what? So if the false positives were evenly distributed across races, then that's no big deal?
  • vkou 8y ago
    Unfortunately, in the current political climate, yes. More specifically, a smaller group of people would care about that, then the group of people that cares about the current, non-even mis-identification, so there would be less political will in opposing it. Cigarettes are considered harmful to everyone, you don't magically get the ability to resist addictive, harmful substances when you turn 21, yet we ban sale of them to minors - for similar reasons.
  • bilbo0s 8y ago
    >were evenly distributed across races, then that's no big deal?... These are the perverse incentives when the standard is "equal treatment under the law", or "equal protection under the law". If they can find some way to treat everyone in a crappy fashion, the law doesn't have a problem with that. The successful class actions tend to come only when it can be shown that you are only treating some people crappy. That would be fairly easy to show in this case. Simply have random whites and blacks stand in front of a camera running MS software and compare the accuracy rates. (Come to think of it, a demo like that would be pretty powerful in a courtroom too. Maybe there is more to this than just PR MS is worried about?) EDIT: I just realized how bad that sounded. To be clear, I'm not saying we should treat some people unfairly. I'm saying the law doesn't care whether we treat people fairly or unfairly, so long as they all get treated the same. I was positing that maybe it would be better if we could try to make the law incentivize treating everyone fairly.
  • laughinghan 8y ago
    I'm confused by the premise of your question. Are under the impression that it's possible for a law enforcement tool to have zero false positives? Nothing has ever had nor will ever have zero chance of false positives. Even DNA evidence can have false positives. Law enforcement, the justice system, and society overall has always had and will always have (highly imperfect) mechanisms to deal with nonzero false positive rates. Therefore, it is of course important that the false positive rate not be higher than average for a subgroup, as they will be disproportionately affected, since our systems will be set up to deal with the lower average false positive rate.
    • SilasX 8y ago
      No, that wasn't my point, and I agree that false positives are inevitable. My point was that you shouldn't only care about an excessive false positive rate when it happens to minority group.
  • PeterStuer 8y ago
    In short: yes. Longer form: Algorithms 'discriminate' (as in differentiate) because that is exactly the job they are tasked with. Is this a picture of a person on list A or not? Is this a picture of criminal activity X? They discriminate on a large number of, often hidden, unknown or not understood features in the data. In many countries the laws dictate that some features such as race, religion, sexual orientation, ... are protected, as in not legally allowed to be used in differentiation. (Take note that in many countries certain national security/safety related organizations are exempt from certain regulations). The models that are used in facial recognition rely (in part) on 'sub-symbolic' probabilistic feedback systems, that in many cases defy post rationalization: we do not have a convincing or specific 'story' about how 'the machine' decides in each cases. This means we can not deductively prove that the above mentioned 'illegal' forms of discrimination were not used (note that it is not sufficient to show that e.g. 'race' or 'gender' was not explicitly used as a feature in the input, as it could be strongly correlated with other inputs or derivations thereof (e.g. type of shampoo bought, zip-code, food preference, ...) So we rely on things like testing post training deployed model to 'vet' the systems aren't biased in the ways we by law and regulation have deemed that they should not be. We test whether the output distribution shifts when we only feed in males vs a mixed gender test set etc. In practice this means that in compliance testing we replace deductive reasoning with correlation. We accept that this will yield false positives, but this choice is partly due to technical limitations (apart from a few well published cases, understanding and explaining how e.g. a deep learning derived model actually 'works' in a rational synoptic way is still beyond us), and part due to ideological stances we have come to. So, yes, we choose to accept false positives that are presented as evenly distributed across specific protected groups or features, while not 'making a fuzz' over others. These are inherently cultural, political, moral and empathic decisions, not 'logical' ones.
    • SilasX 8y ago
      Great, but my objection was to the mindset of: 1) "The algorithm has a 50% false positive rate over everyone" -> awesome, no problem! 2) "The algorithm has a 50% false positive rate of people of color, and a .01% false positive rate over everyone else." -> yikes, we can't use this thing!