First, love the presentation. I wish more papers had such a nice, succinct presentation of the main arguments. Second, a disclaimer, I did not read the full pa

First, love the presentation. I wish more papers had such a nice, succinct presentation of the main arguments. Second, a disclaimer, I did not read the full paper but only went through the link here. The site should be more specific that it refers to the use of predictive optimization in social setups. I thought I was about to read a finding in how predictive optimization is fundamentally flawed, but that is far what this presents. Even within the social scope, the argued flaws seem to be more like a list of things to check against, rather than fundamental flaws: - Good predictions may not lead to good decisions: In a setup where the prediction model is trained independently than the optimization one, this should be obvious. How you craft the optimization model and what you feed it is what matters. - It's hard to measure what we truly care about: This is true in way too many cases, and still we have to do something. We constantly seek proxies to be able to act instead of raising our hands and give up. - The training data rarely matches the deployment setting: A well known issue in ML, not really insightful and just something that should be taken into account when developing models. - Social outcomes aren’t accurately predictable, with or without machine learning: This way too broad a claim. The variance from person to person may make it hard to predict a specific outcome, but it may be possible to provide a model that gives a distribution over outcomes in an accurate way. For example, if you are born in income percentile p, it is hard to predict _exactly_ the income percentile your kids will be born into, but it is feasible to predict a _distribution_ over p. Moreover, if social outcomes were truly random, that is we can't say anything about the future, then there is zero signal between our interventions and the outcomes down the line... which again sort of implies we should never do anything. - Disparate performance between groups can’t be fixed by algorithmic interventions: Again, not a fundamental flaw. There's nothing an algorithmic intervention that makes it inherently biased, rather it will simply reflect the design of the creator. This is a known issue and there is plenty of literature on the matter. - Providing adequate contestability undercuts putative efficiency benefits: This is probably the best argument of the list. The design of predictive optimization models should take into account that they may need to be explained on individual cases, which probably reduces the model space. Or we should get better at explaining complex models! - Predictive optimization doesn't account for strategic behavior: Again, this is just something to take care when designing these systems, not a fundamental flaw. This afflicts human interventions as well, as the examples show. I worry that people in public policy read papers / summaries like this, take it as absolute truths, and then sentence our systems to be fully manual for the foreseeable future.