If computer systems are getting the wrong conclusion, that's a boring philosophical question. Find better SVM params/neural network architecture/etc and you'll

If computer systems are getting the wrong conclusion, that's a boring philosophical question. Find better SVM params/neural network architecture/etc and you'll fix it. But what if the only way to eliminate "institutionalized bias" from the system is to add statistical bias? What happens if "popular prejudice" isn't actually prejudice but merely a useful prior, and one which non-racist [1] machine learning systems faithfully reproduce? [1] I'm using "racist" in the classical sense of f(x.copy(race=A)) != f.copy(x.copy(race=B)), not in the modern sense of "any probabilistic cause for unequal statistical outcomes".