> Say you have an AI that predicts probability of re-offending for prospective parolees. Already you've made one of the most common mistakes in this area: conf

> Say you have an AI that predicts probability of re-offending for prospective parolees. Already you've made one of the most common mistakes in this area: confusing the label for the latent truth. We don't measure "re-offending", we measure rearrest and re-incarceration, which as I've discussed elsewhere are biased (in the sense of "unjust"). > If we were to manually adjust the model to make it display equal risks for both men and women then that is not elimination of bias. It is deliberate introduction of bias to make the output fit our ideal worldview. Again, you are using the word "bias" in a vague and meaningless way. In this context, our goals are constrained by the law - in the US we guarantee people certain rights, and prohibit certain kinds of discrimination. In this example, it would indeed be wrong (and illegal) to punish men more harshly just for being men, since gender is a protected class. In the same way, it would be wrong (and illegal) to punish black people more harshly just for being black. In my state, whites and blacks use cannabis at equivalent rates, and black people are arrested 8 times as often per capita - an algorithm looking at arrests will see that and make the problem worse. You've staked out some sort of naiive and bizarre "AI Purity" stance which completely ignores how such models work, and which misunderstands how we use models to learn things and solve problems. You're also mixing up different definitions of "bias". Some facts which you might find interesting: * Penalization/shrinkage, which is ubiquitous and incredibly useful in many predictive models, is the deliberate introduction of bias to improve prediction performance * Adjusting for confounding in regression/classification type models is one way of accounting for or removing bias in a statistically rigorous way - as you (imprecisely and inaccurately) say, "modifying the model to make it display equal risks". This allows us to measure effects, conditional on known risks or existing patterns. * All statistical and machine learning algorithms suffer from sampling bias, where the data we observe doesn't match the reality in important ways - a fact which you're completely ignoring. If a model reflects the bias in a bad sample, why on earth should we accept that?