3 ms·

How is the existence of a model amplifying any bias? The example the author gives is that conditioning a linear model on some "biased" variable gives a more ac

How is the existence of a model amplifying any bias? The example the author gives is that conditioning a linear model on some "biased" variable gives a more accurate model that predicts large differences between these groups. But the large difference in the groups is right there in the data. It's not amplified in any way. And then somehow this is the modeller's fault for choosing a naughty variable, as if the output of a model has anything to do with the world the modeller wishes existed. The author's beef should be with the idiots who would take this model and then say "yep looks like since these two groups have different regression lines it's totally great that we see large differences between these groups"

sbpayne 6y ago
I was mostly referring to this bit from the article: “Notice how the model that has the lowest mean squared error is also the model that causes the most bias between groups for higher skill levels. This increase in bias cannot be blamed merely on the data. It’s the choice of the model that increases this bias which is the responsibility of the algorithm designer.” My reading of the plots is that the difference between the groups seems to grow beyond what is present in the data. Do you not think this is amplified? As for conditioning on a biased variable; this post uses a contrived example for sure. But the same thing happens with variables that correlate with the “naughty” variable (perhaps author should have explicitly showed this to drive the point?) Removing all variables that correlate with “naughty variables” is a pretty difficult task itself without having something to detect the correlation and tell you. At least that’s been my experience.
- mr_toad 6y ago
  > This increase in bias cannot be blamed merely on the data. This is debatable in this case. The grouped model might be a better fit to the data: it indicates a higher bias in salaries at higher skill levels. It sounds completely plausible that the gender bias for unskilled workers is less than for high skilled workers. Models can be under-fit as well as over fit. Just using a dummy variable for gender might not properly capture interactions like this. But of course it’s always possible that this is just an artefact of the training data.

sbpayne 6y ago

I was mostly referring to this bit from the article: “Notice how the model that has the lowest mean squared error is also the model that causes the most bias between groups for higher skill levels. This increase in bias cannot be blamed merely on the data. It’s the choice of the model that increases this bias which is the responsibility of the algorithm designer.” My reading of the plots is that the difference between the groups seems to grow beyond what is present in the data. Do you not think this is amplified? As for conditioning on a biased variable; this post uses a contrived example for sure. But the same thing happens with variables that correlate with the “naughty” variable (perhaps author should have explicitly showed this to drive the point?) Removing all variables that correlate with “naughty variables” is a pretty difficult task itself without having something to detect the correlation and tell you. At least that’s been my experience.

mr_toad 6y ago
> This increase in bias cannot be blamed merely on the data. This is debatable in this case. The grouped model might be a better fit to the data: it indicates a higher bias in salaries at higher skill levels. It sounds completely plausible that the gender bias for unskilled workers is less than for high skilled workers. Models can be under-fit as well as over fit. Just using a dummy variable for gender might not properly capture interactions like this. But of course it’s always possible that this is just an artefact of the training data.