I agree the parent poster was using the ethical/social sense of "bias" in calling a discriminatory loan risk AI biased, but I don't think it's overfitted and so

I agree the parent poster was using the ethical/social sense of "bias" in calling a discriminatory loan risk AI biased, but I don't think it's overfitted and so I don't think this is what the original poster meant. In the loan example, the objective is to assess risk. Assuming the data we feed it draws equally from all races, the AI will still almost certainly infer a racial component to loan risk. This is not a statistical or systematic error, this is an accurate assessment of society due to long-standing racial inequalities. I think the original poster meant that ethics is not an objective science, and so our preference for certain ethical reasoning is itself a bias. "Correcting" the "biased" outcomes we don't like that are apparent in the data, and that are a reflection of society at large, involves biasing the training data.