3 ms·

If you take the recent anthropic misuse report seriously, they don't need to "take over weapons" because they're actively being used to give a lift in capabilit

If you take the recent anthropic misuse report seriously, they don't need to "take over weapons" because they're actively being used to give a lift in capability to groups who already want to use weapons. > The six cases in this section are divided into two parts. Part I covers four cases in which the actor in question used Claude to develop software for weapons themselves: a guided rocket program, in which the actors conducted a live field test; a design and proposal work on a system to intercept torpedoes; software for a drone swarm, tested in simulation, with its code loaded onto real boards; a targeting software for electronic warfare and for suppressing air defenses. https://www.anthropic.com/threat-intelligence-report-september-2026 https://www.anthropic.com/threat-intelligence-report-septemb...

CM30 12d ago
Yeah, I think this is a bigger threat, at least for the moment. It's not automated systems causing havoc in themselves, it's hostile actors using tech to have outsized impacts on society. As you point out, AI lets criminals and hostile regimes get access to a level of technical knowledge that was previously reserved for larger and more well funded organisations and nation states, that required experts with years of experience to understand. Things like drones and missiles are available to people that would otherwise be incapable of building them. And it's not just limited to that really. Online scams and fraud are arguably easier to set up than ever, given you can use AI to make fake sites for phishing services, write somewhat convincing sounding emails or impersonate the voice of a target's friend or relative to trick someone into sending money. Plus, any weakness in a system that a sentient AI could theoretically exploit can be exploited by a criminal or hostile regime right now. For all the talk of an AI taking over a weapon system, the bigger threat is a cartel, terrorist group, cult, lone nutcase criminal or dangerous regime doing it.

CM30 12d ago

Yeah, I think this is a bigger threat, at least for the moment. It's not automated systems causing havoc in themselves, it's hostile actors using tech to have outsized impacts on society. As you point out, AI lets criminals and hostile regimes get access to a level of technical knowledge that was previously reserved for larger and more well funded organisations and nation states, that required experts with years of experience to understand. Things like drones and missiles are available to people that would otherwise be incapable of building them. And it's not just limited to that really. Online scams and fraud are arguably easier to set up than ever, given you can use AI to make fake sites for phishing services, write somewhat convincing sounding emails or impersonate the voice of a target's friend or relative to trick someone into sending money. Plus, any weakness in a system that a sentient AI could theoretically exploit can be exploited by a criminal or hostile regime right now. For all the talk of an AI taking over a weapon system, the bigger threat is a cartel, terrorist group, cult, lone nutcase criminal or dangerous regime doing it.