This is one of the things that concerns me about things like CIWS/Phalanx, which is a short-range turret gun system for navy vessels. It has a mode which, when

This is one of the things that concerns me about things like CIWS/Phalanx, which is a short-range turret gun system for navy vessels. It has a mode which, when enabled, says "if you show up on my radar and do not have a friendly beacon on you, you are going to get a kilo of tungsten through your center of mass and we're not going to bother asking questions of any human first." The use case is to prevent another Cole incident, but configuring a weapons system in such an autonomous mode means that you're in a mode where unintended deaths are acceptable, and that makes me very, very concerned.