4 ms·

Another problem in the same vein is that when ML algorithms make use of Bayesian inference they can bake in correlations ( e.g. between race and credit score) t

Another problem in the same vein is that when ML algorithms make use of Bayesian inference they can bake in correlations ( e.g. between race and credit score) that we would normally purposefully avoid using as a factor, because while it enhances predictive power, it again codifies our existing biases, prejudices, and injustices. For example if you were deploying an ML model to determine whether someone deserved a loan, features such as ZIP code or race could encode discrimination into the model

CoongLiu 9y ago
By definition, any model is basically bound to be discriminatory. Taking data, extracting common key features, and discarding the rest is essentially generalisation. But the model is amoral. It's (morally) neither good nor bad for utilising certain features. If it turned out that race was the most accurate attribute for a particular situation, it would be nonsensical ignore it. The current trend of trying to paper over biases, while generally born of noble sentiment, probably only perpetuates the problem. Because it's usually done far down stream, and doesn't necessitate change at the source. Functionally, it's like a cover up by a large corporation
- ufo 9y ago
  > If it turned out that race was the most accurate attribute for a particular situation, it would be nonsensical ignore it. Only if you are optimizing for prediction accuracy. If you want to optimize for something like "justice" or "citizen wellbeing" then you might want to come to a different conclusion
maxerickson 9y ago
Including race as a parameter should reduce the impact of those correlations on the output of the model (by allowing the model to measure and control for the bias that exists in the input data). Incautiously using race just because it reflects those existing biases would be a problem. This is what lots of humans do, overestimating the information provided by their own inferences of race. Like internet assholes who blather about how it is rational to be afraid of black men because of their higher rates of assault. Never mind that the absolute rate is still so low that there is ~0 predictive power from the race of a given individual.
- azernik 9y ago
  That's actually a very interesting idea for a technique. But not one I've ever heard of actually being used.
  - maxerickson 9y ago
    It's not a technique, it's a natural outcome of statistical modeling. It doesn't get used because people are innumerate and reactionary.
    - azernik 9y ago
      The technique I'm referring to is including race as a factor, and then instructing your model to disregard that factor. Not being race-blind, but being race-aware while attempting to be neutral to race.
      - opportune 9y ago
        Yeah I think this is an interesting idea but I don't actually understand how the great-grandparent comment believes that this could be done. I don't think it works in a strict Bayesian sense. You would have to go out of your way to instruct your model to operate correctively

CoongLiu 9y ago

By definition, any model is basically bound to be discriminatory. Taking data, extracting common key features, and discarding the rest is essentially generalisation. But the model is amoral. It's (morally) neither good nor bad for utilising certain features. If it turned out that race was the most accurate attribute for a particular situation, it would be nonsensical ignore it. The current trend of trying to paper over biases, while generally born of noble sentiment, probably only perpetuates the problem. Because it's usually done far down stream, and doesn't necessitate change at the source. Functionally, it's like a cover up by a large corporation

ufo 9y ago
> If it turned out that race was the most accurate attribute for a particular situation, it would be nonsensical ignore it. Only if you are optimizing for prediction accuracy. If you want to optimize for something like "justice" or "citizen wellbeing" then you might want to come to a different conclusion

maxerickson 9y ago

Including race as a parameter should reduce the impact of those correlations on the output of the model (by allowing the model to measure and control for the bias that exists in the input data). Incautiously using race just because it reflects those existing biases would be a problem. This is what lots of humans do, overestimating the information provided by their own inferences of race. Like internet assholes who blather about how it is rational to be afraid of black men because of their higher rates of assault. Never mind that the absolute rate is still so low that there is ~0 predictive power from the race of a given individual.

azernik 9y ago
That's actually a very interesting idea for a technique. But not one I've ever heard of actually being used.
- maxerickson 9y ago
  It's not a technique, it's a natural outcome of statistical modeling. It doesn't get used because people are innumerate and reactionary.
  - azernik 9y ago
    The technique I'm referring to is including race as a factor, and then instructing your model to disregard that factor. Not being race-blind, but being race-aware while attempting to be neutral to race.
    - opportune 9y ago
      Yeah I think this is an interesting idea but I don't actually understand how the great-grandparent comment believes that this could be done. I don't think it works in a strict Bayesian sense. You would have to go out of your way to instruct your model to operate correctively