See https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing https://www.propublica.org/article/machine-bias-risk-assessm... for

See https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing https://www.propublica.org/article/machine-bias-risk-assessm... for a well-known example in this space. This article: https://www.nature.com/articles/d41586-018-05469-3 https://www.nature.com/articles/d41586-018-05469-3 also highlights the issues pretty well IMO. It's not just about letting the data speak. The data is gathered by someone, containing historical bias we might nowadays disagree with, models are chosen by people, using parameters set by people, using evaluations set by people. It's about making sure models are both predictive, usable, and fair.