I think the desirability of eliminating this kind of bias depends on the context of the problem that the algorithm would be used to solve. Per your example, sup

I think the desirability of eliminating this kind of bias depends on the context of the problem that the algorithm would be used to solve. Per your example, suppose that strength is what an algorithm is predicting. We might not want the algorithm to lowball the strength of a candidate just because that candidate "looks like a woman" due to some other causally unrelated correlation.