4 ms·

> If bias increases accuracy/precision, it's not bias. Yes it is. It's bias in your fitness function. Accuracy and precision are not handed down by the gods.

> If bias increases accuracy/precision, it's not bias. Yes it is. It's bias in your fitness function. Accuracy and precision are not handed down by the gods. We write the functions that evaluate our models, and it's our job to make sure that the values they promote match up with the real-world outcomes we desire, and to constantly monitor and re-evaluate those outcomes. Fancier machine learning techniques will never be able to avoid Goodheart's Law: "Any measurement, no matter how reliable, when regarded as a target, ceases to be a good measurement."

crowdpleaser 8y ago
But our 'fitness functions' for social problems are normally pretty good or above reproach of the model. These tend to be easy to measure like 'did the person skip bail?' If a model of 'likelihood to show up to court after making bail' can make better predictions with information about protected characteristics (e.g if the model used sex to predict likelihood to show up in court), that feature would reduce the bias of the model. I think the issue progressives have with 'bias' is that some of society's prejudices ('bias') have an evidentiary basis. We already make decisions that progressives would tell us are prejudiced but we probably want to use those prejudices if they're useful. Consider a group of young men standing outside of a Church. If they're all clean shaven, smiling, and 'appropriate' for the Church it's nothing concerning. If they're white guys with shaved heads / neo nazi haircuts, and they don't look nice, and they're standing outside of a black Church, the prejudiced among us might correctly decide to alert the authorities to it. My personal opinion is that we should allow models to consider protected traits but we should ensure that models that make important decisions aren't prejudiced along those protected traits. The way to measure this is simply to ensure that the accuracy and precision of the classification decisions are comparable among protected traits.
Veelox 8y ago
Here is a somewhat modified situation. Say you use a machine learning algorithm to recommend if a person currently accused of a crime should be allowed bail. The fitness function should be to maximize the number of people who are allowed bail and minimize the number of people who miss their court date. Say that the model allows X% of people to have bail and makes sure only Y% fail to show up. The model then adds race as an input to the model. This improves the model so that it allows X+5% of people to have bail and makes sure that only Y/2% fail to show up. It also has the effect that it increases the chance that a black person is denied bail and increases the chance that a white person is allowed bail. Do you think the inclusion of race is bias? Should race be removed from the input to the model?

crowdpleaser 8y ago

But our 'fitness functions' for social problems are normally pretty good or above reproach of the model. These tend to be easy to measure like 'did the person skip bail?' If a model of 'likelihood to show up to court after making bail' can make better predictions with information about protected characteristics (e.g if the model used sex to predict likelihood to show up in court), that feature would reduce the bias of the model. I think the issue progressives have with 'bias' is that some of society's prejudices ('bias') have an evidentiary basis. We already make decisions that progressives would tell us are prejudiced but we probably want to use those prejudices if they're useful. Consider a group of young men standing outside of a Church. If they're all clean shaven, smiling, and 'appropriate' for the Church it's nothing concerning. If they're white guys with shaved heads / neo nazi haircuts, and they don't look nice, and they're standing outside of a black Church, the prejudiced among us might correctly decide to alert the authorities to it. My personal opinion is that we should allow models to consider protected traits but we should ensure that models that make important decisions aren't prejudiced along those protected traits. The way to measure this is simply to ensure that the accuracy and precision of the classification decisions are comparable among protected traits.

Veelox 8y ago

Here is a somewhat modified situation. Say you use a machine learning algorithm to recommend if a person currently accused of a crime should be allowed bail. The fitness function should be to maximize the number of people who are allowed bail and minimize the number of people who miss their court date. Say that the model allows X% of people to have bail and makes sure only Y% fail to show up. The model then adds race as an input to the model. This improves the model so that it allows X+5% of people to have bail and makes sure that only Y/2% fail to show up. It also has the effect that it increases the chance that a black person is denied bail and increases the chance that a white person is allowed bail. Do you think the inclusion of race is bias? Should race be removed from the input to the model?