> In June 2015, for example, Google’s photo categorization system identified two African Americans as “gorillas.” > Law enforcement officials have already been

> In June 2015, for example, Google’s photo categorization system identified two African Americans as “gorillas.” > Law enforcement officials have already been criticized, for example, for using computer algorithms that allegedly tag black defendants as more likely to commit a future crime, even though the program was not designed to explicitly consider race. Consider these two examples from the article. One is when the machine failed in a particular undesirable fashion. The other is when the machine "worked^" but in an undesirable fashion. (^That is, overall it might have worked well, but the bias in errors wasn't sufficiently explored.) Modelling "undesirable fashion", which is a dynamic, subjective social construct, is far more difficult than either of the tasks originally set in the examples. In the first example, how many images of people were confused with non-humans? I can't believe this was the only failed result. The only reason this particular example was problematic was that these specific people were specifically identified as gorillas. No problem if they were identified as a car, or a chess piece, or a satellite. Gorillas though, that's a special failure. We all know why. And we know why the error occurred. If the machine sees predominantly white people and gorillas, then "people" are white and "almost people" but with darker pixels is "gorilla". It is human prejudice in interpreting the results that made the error an issue.