>When that happens, we identify why the interaction is problematic, why the human is not trusting the machine, or why the way that the data is presented is bein

>When that happens, we identify why the interaction is problematic, why the human is not trusting the machine, or why the way that the data is presented is being misunderstood — and then we change those things. Such changes should make us question to what extent human oversight of algorithms is truly meaningful. Nonsense; this is like saying "we changed the wheel on the bike so that the rider didn't fall off it so often which makes us question if they can ride the bike". If the interface is being changed to better deceive a user then that of course is problematic, but if it is being changed to allow the user to make accurate decisions then that is the function of the system. >But now imagine a system similar to HARPY, a “fire-and-forget” system that targets enemy combatants in an area or enemy ships (to use an example that minimizes the collateral risk). Proponents of meaningful human control do not believe it should ever be left up to a machine to target humans. Strawman - that is not what "Proponents of meaningful human control" believe, I am sure you can find some people who say that, but then they will also say that they don't believe that a bullet should be fired or a bomb dropped. Machines target humans all the time in the sense that unintended collateral damage occurs whenever bombs are dropped. The humans doing it must accept responsibility for this; that is the same for more sophisticated processes - the humans remain on the hook for unleashing the device. If the device decides to slaughter a primary school class that's on you in the same way as it is if you target a school yourself. >I think there are good reasons to question this. Specifically, I am not persuaded that there is a significant moral difference between an operator identifying a single human target based on some data and a human operator drawing a box and defining targets within that box based on equally good data. Which shows that you, dear writer, should never be allowed to decide a god damn thing. There is a chasm of uncertainty between the two scenarios, and a chasm of indirection. >whether in cases when the system works better or as well without a human operator, we have moral reasons to place a human in the loop. Because we live in an open world and the judgement of performance is post hoc, you can't know that it will work until it's done its thing. The judgement has to be when the act is commissioned because the consequences of the act cannot be reversed.