This is nonsensical. Already a few years ago the USAF IIRC ran some tests in which the AI first bombed the control tower so humans couldn't call it off from its

This is nonsensical. Already a few years ago the USAF IIRC ran some tests in which the AI first bombed the control tower so humans couldn't call it off from its mission, thereby increasing its pass rate. The whole point of this is they do things an unintended ways. And that's potentially devastating given their persistence & hacking skillz. Also you're using the hosted versions that sit behind their guardrails when you use OpenAI/Anthropic APIs.