3 ms·

I'll grant you that there weren't much ethical considerations to speak of in the last world wars, and perhaps my naive view has been unrealistically colored by

I'll grant you that there weren't much ethical considerations to speak of in the last world wars, and perhaps my naive view has been unrealistically colored by sci-fi TV shows, but my point is simple: a broken autonomous killing system won't know when to stop. Even unethical war makers have affinity to humankind and know when to stop. And despite the meaning implied by the MAD doctrine, it never meant that humans will nuke themselves to oblivion; it simply says a nash equilibrium will be achieved with both conflicting parties staying out of each other's way.

vidarh 7y ago
Unintended consequences certainly gets more and more dangerous the more advanced autonomy these weapons gets. In keeping with your mention of sci-fi, there's a Star Trek Voyager episode where two sides of killer robots continue to wage war against each other after having killed their makers because their makers had agreed on a truce, and their instructions had been to eliminate anyone that prevented them from beating their enemy and hence by agreeing a truce their makers had become an obstacle to achieving their programmed goal. Having done software development my entire life, I'd never trust anyone to manage to avoid every pitfall in guaranteeing you have a foolproof way of making them stop...