I initially rolled my eyes at the title, but then the subtitle adds “On the Legitimacy of...” and the rest of the article raises good points, so it’s just the m

I initially rolled my eyes at the title, but then the subtitle adds “On the Legitimacy of...” and the rest of the article raises good points, so it’s just the misleading title I dislike. I don’t think there is a good argument against predictive optimization itself, but that’s not what the article is about—rather it is about how we are implementing it incorrectly and misusing its results. I’ve posted on here before that I think we as a society need to carefully distinguish between predictive ability, causal inference, and ethical policies. You can predict things without understanding the cause. And you can determine the cause of something without being able to make future predictions about it. But neither causation nor prediction are sufficient to inform an ethical policy. A simple example might be a genetic abnormality that causes disease. Suppose we can use someone’s DNA to predict early in life and with high accuracy whether they are likely to eventually develop a disease that entails exorbitant medical costs. And suppose we can even identify the combination of genes that fully determines the outcome of whether a person will develop the disease or not. Health insurance companies might decide it’s not in their financial interest to insure these people, at least not without significantly raising the affected customers’ premiums. But as a matter of morality, society decides that because this disease is not the fault of any individual or within any individual’s ability to control, insurance (and subsequently, all customers) should accept this additional cost as a necessary loss in order to ensure fairness. To expound on the article’s other point, what is frequently claimed as “predictive” often is not actually so. As a data scientist, one of the things that I’ve noticed within my industry that is really glossed over is evaluating the accuracy of accuracy evaluation. Determining the quality of a model’s predictions is to some degree extremely difficult to do correctly. As a motivating example, consider a collection of models from different seismologists for earthquake prediction. Could you easily tell me which model is “the best” and to what extent each model is wrong? Any ethical policy that includes predictive ability as one of its components should require intense scrutiny of the quality of the predictions. Not to mention that policies that depend upon predictive ability should already be extremely rare—I can think of perhaps one example: “what value of n to use for Blackstone’s ratio?” In this case, the true value of n (as opposed to our desired value for n) depends on how accurately we can predict whether someone is legally guilty or innocent. And even in that case, we never have ground truth data, so our ability to determine precision/recall is fundamentally limited.