4 ms·
I think that if you read their list of "Flaws of predictive optimization" as a primer for what developers of automated advisory or decision making systems ought
by walnutclosefarm 4y ago
I think that if you read their list of "Flaws of predictive optimization" as a primer for what developers of automated advisory or decision making systems ought to be thinking about, and ultimately be accountable for, then this is very useful. But then concluding that we should be "Against predictive optimization)" (Their choice of headline), and provide a gauntlet of 23 ways to de-legitimize a predictive algorithm (as in their linked rubric), all of which, in all likelihood, no algorithm can survive, it goes way over the top.
Predictive optimization is a necessary fact of life in many functions. The question is often not "should we do it," but "should we do it with a particular algorithm, in a particular setting?" The fundamental questions we should be asking, from the top down:
1) does making the prediction systematically, and well, serve a useful, desirable social purpose. That is, do accurate predictions actually do us a net good? This can be a hard question - using predictive algorithms to screen for adverse futures, given that all predictions, including human-based decisions, throw both false positives and false negatives, you have to weight whether the harm in false predictions is outweighed by the good in true predictions.
2) Does a machine algorithm improve on human judgement in the question and system in question? Machine algorithms don't have to be perfect to be better than human judgement, which is often abysmal when viewed on a system-wide basis.
3) Does the system into which the algorithm is being deployed provide reasonable mechanisms for detection, recourse and compensation for false predictions? Because, again, there will be false predictions.
A lot of the use of AI in social decision making that the authors criticize would flunk these questions. Not all, though.