4 ms·

A lot hinges on how you define fairness. Is it unfair if a higher percentage of <racial group> is flagged as “criminal activity risks”, to use your example, tha

A lot hinges on how you define fairness. Is it unfair if a higher percentage of <racial group> is flagged as “criminal activity risks”, to use your example, than of <other racial group>? What if it aligns with reality? No matter how accurate your metric is—nay, because of its accuracy—people will be mad at you because it reflects an underlying reality they are in denial of or believe is itself unfair, and want to rectify by requiring fictions elsewhere. I don’t envy people working on this because you by definition can’t win. Some of the powerful political forces, on the one hand, and the demands for accuracy and effectiveness, on the other, are irreconcilable.

yorwba 8y ago
> Is it unfair if a higher percentage of <racial group> is flagged as “criminal activity risks”, to use your example, than of <other racial group>? What if it aligns with reality? The article is about the unfairness of a higher percentage of <racial group> getting incorrectly flagged as “criminal activity risks”. The fairness of the expected output is essentially out of scope for this kind of research, it's all about the distribution of correct vs. erroneous decisions. The underlying reality is an imbalance in data quality or similar, not whatever you were thinking about.
jakelazaroff 8y ago
That's why the grandparent mentioned feedback. Even if a disparity "aligns with reality", if the algorithm is used in a way that reinforces the disparity, then it's discriminatory. When designing algorithms as parts of systems, we need to be careful to not ossify statistics we're ostensibly just reflecting.