4 ms·

I've seen many arguments for why AI will be safe. Yours relies on the assumption that no one would be stupid or reckless enough to give an AI that might be much

I've seen many arguments for why AI will be safe. Yours relies on the assumption that no one would be stupid or reckless enough to give an AI that might be much more capable cognitively than a person the ability to manufacture weapons it has designed or at least no one would be stupid enough to do the final assembly of anything that might be a weapon from parts the AI designed and arranged to be made. One argument (many years ago) was that no one would be stupid enough to give such an AI access to the internet or the ability to run code it has written. (In the HuggingFace incident, the AIs were given "read access" to the internet, or at least that was OpenAI's intent, but the AIs escalated that to write access.)

sarchertech 18d ago
> at least no one would be stupid enough to do the final assembly of anything that might be a weapon from parts the AI designed and arranged to be made. Worrying that an AI is going to figure out how to make a device that looks to everyone investigating it like a power generator, but once turned on kills all humans, is like worrying that an AI might discover magic. I can’t prove it’s not possible, but I’m not spending time worrying about it. As weapons get more and more powerful they have always gotten more and more complex and required more and more time, space, money, and infrastructure to build and deploy. There’s no reason to think that this will change with AI.