Even with no bias, uncertainty can cause problems. Say an ML system is tasked with finding the top 10 candidates in terms of "confidence that they will be able

Even with no bias, uncertainty can cause problems. Say an ML system is tasked with finding the top 10 candidates in terms of "confidence that they will be able to do the job". Then if it has little training data on candidates of a particular class, and those in that class are actually quite different on many different variables (so it can't generalize very well), it may not be able to reach the required levels of confidence for them. I think this is actually the reason for a lot of accidental discrimination, because human judges would have exactly the same problem. I remember in school, playing chess a couple of times against a guy fresh from Sudan. He had the most unsettling smile, and played very unorthodox openings. I won some, he won some - but I always suspected he was stronger than me, and just being polite/testing me. It's just impossible to read someone from a culture so different. I'm glad we didn't play poker, to put it like that.