> Because that's basically what we mean by super-intelligence, the ability to reach any conceivable goal. I appreciate this formulation as it allows me to fina

> Because that's basically what we mean by super-intelligence, the ability to reach any conceivable goal. I appreciate this formulation as it allows me to finally articulate my core disagreement with AI doomers: it's a fantasy, and a surprisingly common one, that is secretly supremacist. It's similar to the "9/11 was an inside job" types: they simply refuse to accept that the almighty US could get harmed by a sufficiently determined terrorist cell group, so they invent any number of reasons how the cackling overlords were behind it the whole time, instead of the more boring reality that the US military and intelligence complexes are incompetent and squabbling bureaucracies. With AI it's the belief that human endeavor can finally defeat nature once and for all, by defeating humans themselves. But as your formulation shows, an AI achieving this would have to solve precisely the same problems that power-hungry human organizations do, which is "overcome the other guy". Why do we blithely assume that "AI" would be some unitary entity? We already have bard vs chatgpt vs llama. How are they going to paperclip each other out of existence let alone us? Finally, we don't have to fantasize about immortal superhuman organizations out to destroy humanity, we have them already: multinational corporations :)