4 ms·

Agree, the ants argument is simply a bad one. However it is not a necessary one to argue that superintelligence is dangerous. The whole superintelligence philo

Agree, the ants argument is simply a bad one. However it is not a necessary one to argue that superintelligence is dangerous. The whole superintelligence philosophical argument gets much easier if you replace "super-intelligent AI" with "omnipotent human being". Because that's basically what we mean by super-intelligence, the ability to reach any conceivable goal. Also the concept of "alignment" becomes superfluous: a human being is, by default, "aligned", and yet an omnipotent human being is quite obviously an unacceptable danger. There's an episode of The Twilight Zone that explores this idea. It's called "It's a good life" and it's about an omnipotent 6yo child who has enslaved his entire village, inflicting horrible punishments to those who anger him. It serves as a good reminder of how who we are is also the product of our limits.

andai 3y ago
Another great example is Akira. I think alignment just means "what is good". (If we define aligned as moral, it's easy to disprove that humans are aligned by default.) Humans haven't quite solved morality, so I'm not sure how we intend to teach it. A superhuman intelligence will require superhuman morality. Perhaps if we can create one, we can create the other? But it's not a given by any means.
- throw310822 3y ago
  > If we define aligned as moral, it's easy to disprove that humans are aligned by default What is the quote? "All power corrupts, absolute power corrupts absolutely". I am not sure any human would remain moral once given absolute power.
spopejoy 3y ago
> Because that's basically what we mean by super-intelligence, the ability to reach any conceivable goal. I appreciate this formulation as it allows me to finally articulate my core disagreement with AI doomers: it's a fantasy, and a surprisingly common one, that is secretly supremacist. It's similar to the "9/11 was an inside job" types: they simply refuse to accept that the almighty US could get harmed by a sufficiently determined terrorist cell group, so they invent any number of reasons how the cackling overlords were behind it the whole time, instead of the more boring reality that the US military and intelligence complexes are incompetent and squabbling bureaucracies. With AI it's the belief that human endeavor can finally defeat nature once and for all, by defeating humans themselves. But as your formulation shows, an AI achieving this would have to solve precisely the same problems that power-hungry human organizations do, which is "overcome the other guy". Why do we blithely assume that "AI" would be some unitary entity? We already have bard vs chatgpt vs llama. How are they going to paperclip each other out of existence let alone us? Finally, we don't have to fantasize about immortal superhuman organizations out to destroy humanity, we have them already: multinational corporations :)
- api 3y ago
  That was the South Park take on 9/11 conspiracies: that the government was conspiring to promote them to create the impression they are competent enough to pull it off.

andai 3y ago

Another great example is Akira. I think alignment just means "what is good". (If we define aligned as moral, it's easy to disprove that humans are aligned by default.) Humans haven't quite solved morality, so I'm not sure how we intend to teach it. A superhuman intelligence will require superhuman morality. Perhaps if we can create one, we can create the other? But it's not a given by any means.

throw310822 3y ago
> If we define aligned as moral, it's easy to disprove that humans are aligned by default What is the quote? "All power corrupts, absolute power corrupts absolutely". I am not sure any human would remain moral once given absolute power.

spopejoy 3y ago

> Because that's basically what we mean by super-intelligence, the ability to reach any conceivable goal. I appreciate this formulation as it allows me to finally articulate my core disagreement with AI doomers: it's a fantasy, and a surprisingly common one, that is secretly supremacist. It's similar to the "9/11 was an inside job" types: they simply refuse to accept that the almighty US could get harmed by a sufficiently determined terrorist cell group, so they invent any number of reasons how the cackling overlords were behind it the whole time, instead of the more boring reality that the US military and intelligence complexes are incompetent and squabbling bureaucracies. With AI it's the belief that human endeavor can finally defeat nature once and for all, by defeating humans themselves. But as your formulation shows, an AI achieving this would have to solve precisely the same problems that power-hungry human organizations do, which is "overcome the other guy". Why do we blithely assume that "AI" would be some unitary entity? We already have bard vs chatgpt vs llama. How are they going to paperclip each other out of existence let alone us? Finally, we don't have to fantasize about immortal superhuman organizations out to destroy humanity, we have them already: multinational corporations :)

api 3y ago
That was the South Park take on 9/11 conspiracies: that the government was conspiring to promote them to create the impression they are competent enough to pull it off.