You're right, of course: "human beings cant take the truth and consequences." "That is exactly accurate [...] That's a fact" - both true. I think the issue is l

You're right, of course: "human beings cant take the truth and consequences." "That is exactly accurate [...] That's a fact" - both true. I think the issue is less truth and more action, as in "I believe it to be immoral to act upon that fact in manner X." I can come up with a dozen logical statements that are equally correct and immoral to act upon in certain ways. If an ML algorithm had come up with them, they would be no less correct and no less immoral. For example, "If a woman applicant is pregnant, she will take maternity leave and be less productive year 1. Therefore, we should not hire her." <- likely correct, definitely immoral (imo). COMPAS drew a number of objective conclusions and ties them to specific recommendations in its intended use: (1) if you're poor, you're more likely to recidivate. Therefore, you should get a longer sentence. (2) if you live in a black community, you're more likely to recidivate. Therefore, you should get a longer sentence (3) if you're related to a gang member (not just one yourself - even if your estranged brother joined a gang a decade ago), you're more likely to recidivate. Therefore, you should get a longer sentence. Notice: the "you should get a longer sentence" bit is actually part of the tool. That was the intervention it was designed to suggest. There's a number of critiques of that connection (mathematical, legal, moral), but it's not mine, nor is it the "AI's"; it's Northpointe's. And ultimately that's the scandal & intrigue: the math is right logically, but even so, the tool's recommendation is wrong ethically.