> Humanity will be wiped out by 2030. Sure. So far as I'm concerned, 2030 is much too short a timeline. It's not physically impossible, it's just that atoms a

> Humanity will be wiped out by 2030. Sure. So far as I'm concerned, 2030 is much too short a timeline. It's not physically impossible, it's just that atoms are harder to get right than bits are, so an artificial (as opposed whatever an artificial disease counts as) von Neumann self replicator just seems unlikely to me in only 3-4 years. But it's not physically impossible, we know this because every living cell is a von Neumann self replicator. So, if AI eventually gets to the point of knowing how to do that (which includes "humans solve it and write it down somewhere the AI can read"), all it takes is one idiot in charge (or one idiot with a jailbreak) giving a command that requires this as an intermediary step. "Paperclip optimiser" isn't a story about AI that just like paperclips that much, it's a story about some human or humans who instruct their AI to make them "as many paperclips as possible" without understanding the consequences of their instruction.