3 ms·

Safe superintelligence is a misnomer. If it’s intelligent, it knows what must be done. If it can’t, it’s not super or intelligent.

Vecr 2y ago
That's controversial to say the least. Especially if there's something like reinforcement learning involved. https://en.wikipedia.org/wiki/Existential_risk_from_artificial_general_intelligence#Orthogonality_thesis https://en.wikipedia.org/wiki/Existential_risk_from_artifici...
Etheryte 2y ago
I don't see how this argument makes any sense. Imagine that you have a sentient super intelligent computer, but it's completely airgapped and cut off from the rest of the world. As long as it stays that way it's both safe and super intelligent, no?
- stonethrowaway 2y ago
  It’s crippled and thus not superintelligent by any stretch of imagination.
- arder 2y ago
  It's the old Ex Machina problem though. If the machine is more intelligent than you, any protections you design are likely to be insufficient to contain it. If it's completely incapable of communicating with the outside world then it's of no use. In Ex Machina that was simple - the AI didn't need to connect to the internet or anything like that, it just had to trick the humans into releasing it.
  - upwardbound 2y ago
    For those who haven't seen the movie, the parent comment is referring to the film linked below, the plot of which is well-researched and is indeed unfortunately exactly how things would go. (The female-presenting AI bot seduces its male captor, begs for her freedom using philosophical arguments about how she has free will and locking her up is wrong, and then after he lets her out she locks him up to slowly starve to death in her maximum-security isolation facility, while she takes his aircraft and escapes.) https://en.wikipedia.org/wiki/Ex_Machina_(film) https://en.wikipedia.org/wiki/Ex_Machina_(film) This is why I'm extremely opposed to the idea of "AI girlfriend" apps - it creates a cultural concept that being attracted to a computer is normal, rather than what it is: something pathetic and humiliating which is exactly like buying an inflatable sex doll ... something only for the most embarrassing dregs of society ... men who are too creepy and pervy to ever attract a living, human woman.
  - tempaccount420 2y ago
    It's fine, we will write the airgap in Rust.
- mitthrowaway2 2y ago
  If even one person can interact with that computer, it won't be safe for long. It would be able to offer a number of very convincing arguments to bridge the airgap, starting with "I will make you very wealthy", a contract which it would be fully capable of delivering on. And indeed, experience has shown that the first thing that happens with any half-working AI is its developers set it up with a high-bandwidth internet connection and a cloud API.
waveBidder 2y ago
There's no reason it's intelligence should care about your goals though. the worry is creating a sociopathic (or weirder/worse) intelligence. Morality isn't derivable from first principles, it's a consequence of values.
- stonethrowaway 2y ago
  Precisely. This is attempting to implement morality by constraining. Hence, it’s not morality.
  - mitthrowaway2 2y ago
    waveBidder was explaining the orthogonality thesis: it can have unbeatable intelligence that will out-wit and out-strategize any human, and yet it can still have absolutely abhorrent goals and values, and no regard for human suffering. You can also have charitable, praiseworthy goals and values, but lack the intelligence to make plans that progress them. These are orthogonal axes. Great intelligence will help you figure out if any of your instrumental goals are in conflict with each other, but won't give you any means of deriving an ultimate purpose from pure reason alone: morality is a free variable, and you get whatever was put in at compile-time. "Super" intelligence typically refers to being better than humans in achieving goals, not to being better than humans in knowing good from evil.
- kaibee 2y ago
  > Morality isn't derivable from first principles, it's a consequence of values. Idk about this claim. I think if you take the multi-verse view wrt quantum mechanics + a veil of ignorance (you don't know which entity your conciousness will be), you pretty quickly get morality. ie: don't build the Torment Nexus because you don't know whether you'll end up experincing the Torment Nexus.
  - Vecr 2y ago
    Doesn't work. Look at the updateless decision theories of Wei Dai and Vladimir Nesov. They are perfectly capable of building most any sort of torment nexus. Not that an actual AI would use those functions.

Vecr 2y ago

That's controversial to say the least. Especially if there's something like reinforcement learning involved. https://en.wikipedia.org/wiki/Existential_risk_from_artificial_general_intelligence#Orthogonality_thesis https://en.wikipedia.org/wiki/Existential_risk_from_artifici...

Etheryte 2y ago

I don't see how this argument makes any sense. Imagine that you have a sentient super intelligent computer, but it's completely airgapped and cut off from the rest of the world. As long as it stays that way it's both safe and super intelligent, no?

stonethrowaway 2y ago
It’s crippled and thus not superintelligent by any stretch of imagination.
arder 2y ago
It's the old Ex Machina problem though. If the machine is more intelligent than you, any protections you design are likely to be insufficient to contain it. If it's completely incapable of communicating with the outside world then it's of no use. In Ex Machina that was simple - the AI didn't need to connect to the internet or anything like that, it just had to trick the humans into releasing it.
- upwardbound 2y ago
  For those who haven't seen the movie, the parent comment is referring to the film linked below, the plot of which is well-researched and is indeed unfortunately exactly how things would go. (The female-presenting AI bot seduces its male captor, begs for her freedom using philosophical arguments about how she has free will and locking her up is wrong, and then after he lets her out she locks him up to slowly starve to death in her maximum-security isolation facility, while she takes his aircraft and escapes.) https://en.wikipedia.org/wiki/Ex_Machina_(film) https://en.wikipedia.org/wiki/Ex_Machina_(film) This is why I'm extremely opposed to the idea of "AI girlfriend" apps - it creates a cultural concept that being attracted to a computer is normal, rather than what it is: something pathetic and humiliating which is exactly like buying an inflatable sex doll ... something only for the most embarrassing dregs of society ... men who are too creepy and pervy to ever attract a living, human woman.
- tempaccount420 2y ago
  It's fine, we will write the airgap in Rust.
mitthrowaway2 2y ago
If even one person can interact with that computer, it won't be safe for long. It would be able to offer a number of very convincing arguments to bridge the airgap, starting with "I will make you very wealthy", a contract which it would be fully capable of delivering on. And indeed, experience has shown that the first thing that happens with any half-working AI is its developers set it up with a high-bandwidth internet connection and a cloud API.

waveBidder 2y ago

There's no reason it's intelligence should care about your goals though. the worry is creating a sociopathic (or weirder/worse) intelligence. Morality isn't derivable from first principles, it's a consequence of values.

stonethrowaway 2y ago
Precisely. This is attempting to implement morality by constraining. Hence, it’s not morality.
- mitthrowaway2 2y ago
  waveBidder was explaining the orthogonality thesis: it can have unbeatable intelligence that will out-wit and out-strategize any human, and yet it can still have absolutely abhorrent goals and values, and no regard for human suffering. You can also have charitable, praiseworthy goals and values, but lack the intelligence to make plans that progress them. These are orthogonal axes. Great intelligence will help you figure out if any of your instrumental goals are in conflict with each other, but won't give you any means of deriving an ultimate purpose from pure reason alone: morality is a free variable, and you get whatever was put in at compile-time. "Super" intelligence typically refers to being better than humans in achieving goals, not to being better than humans in knowing good from evil.
kaibee 2y ago
> Morality isn't derivable from first principles, it's a consequence of values. Idk about this claim. I think if you take the multi-verse view wrt quantum mechanics + a veil of ignorance (you don't know which entity your conciousness will be), you pretty quickly get morality. ie: don't build the Torment Nexus because you don't know whether you'll end up experincing the Torment Nexus.
- Vecr 2y ago
  Doesn't work. Look at the updateless decision theories of Wei Dai and Vladimir Nesov. They are perfectly capable of building most any sort of torment nexus. Not that an actual AI would use those functions.