It would be nice to do a well-designed experiment to ask what are the biological differences in different categories of humans (eg old vs young). Then use that

It would be nice to do a well-designed experiment to ask what are the biological differences in different categories of humans (eg old vs young). Then use that ground truth to assess whether an ML model is generating the biases that deviate from the ground truth. That way there is no a priori assumption of what is technical bias and what is real signal. With the level of investment people are putting into ML technologies, these types of ground truth calibrations would cost only a fraction of the total budget.