An AI version of Popehat's Law of Goats applies. If an AI attached to a nuclear command center issues an order to launch the nukes, somebody (likely including u

An AI version of Popehat's Law of Goats applies. If an AI attached to a nuclear command center issues an order to launch the nukes, somebody (likely including us) is fucked. It doesn't matter if the AI "knew" anything, if it deliberated over its target selections, if it assessed and prioritized the most likely threats. Something happened and now we have nukes in the air. If an AI reliably produces good code in response to prompts, it is a better program than most humans. If an AI reliably produces prose that's free of errors, well organized, and summarizes the issues asked for, it is a good writer. Irrespective of what we can say it "knows" or doesn't know.