Ah ok I see what you mean. I think what we have today (weak ML) are models which can be trained to operate at the quality of a human for specific tasks. So I

Ah ok I see what you mean. I think what we have today (weak ML) are models which can be trained to operate at the quality of a human for specific tasks. So I am definitely terrified by the recidivism example, because it’s not obvious to me that there is a human who can do that task, but I’m pretty relaxed about the coverage one (because presumably at least in principle if you automate these sorts of manual tasks you can drive down the cost of providing care - if that saving passed on to consumer). The image recognition one is a bit more complicated but if your scenario is targeting a hellfire missile I’m not sure the human-in-the-driving-seat method has created good outcomes so far, so I’m at least in principle open to some level of AI involvement perhaps short of full autonomy. But I can definitely understand the problem of incentives. Ultimately we have that issue with humans though and it’s in some respects harder to audit them.