3 ms·

The AI Control Problem

8 points by ZephyrBlu 6y ago

ilaksh 6y ago
It is possible to build more general purpose AI without creating digital life that will compete with us. But people will stupidly go ahead and create digital emulations of humans with human-like desires and instincts but superhuman capabilities. When they do that, un-augmented humans will not be in control over the planet. The AIs will be. Maybe the people with Neuralink's or whatever can hope to be their friends. Or favored pets. But there is no way we can control superintelligence unless we explicitly design it to be nothing like people or other animals. It can't have any true autonomy or instincts, open-ended action, or many other capabilities and characteristics of creatures. If we are careful we may be able to stay in control of the planet for another 40 years, 100 years, maybe even 200 if we are lucky. But I think eventually someone will build digital superintelligent creatures that are (human) animal-like and completely autonomous out of a type of moral imperative that puts the evolution of intelligent life over the supremacy of humanity. I hope they wait until after my lifetime though.
deleted 6y ago
[deleted]
aabhay 6y ago
The AI control problem is a sci-fi narrative that is less instructive as a future speculation, and more instructive as an allegory for the present, in which a vanishingly small upper elite power class can construct a false reality for a destitute proletariat.
- ALittleLight 6y ago
  Why do you think that? My own intuition says that AI Control is a huge problem in the literal sense and not a metaphor at all. My reasoning: 1. No known reason why AGI can't exist. We know our general intelligence exists, so unless there is something magic about us, AGI should be possible. 2. Engineered things can surpass biological things on specific dimensions. A jet flies faster than a bird, a crane lifts more than an elephant, car faster than cheetah, etc. If the same concept holds for intelligence, we could build something much smarter than us. Put another way, we don't know how far above us the limit of intelligence is, so there is some unknown space into which our new intelligence may fall. It could be much smarter than us. 3. Recursive self-improvement, digital copies, horizontal scaling, never getting tired, faster physical limits on computation, perfect memory - AGI may have lots of advantages over us. 4. Intelligence more powerful than our own could do anything, including taking over the world, exterminating humanity, creating a virtual hell and trapping everyone in it. The AGI may not have a motive to do so, but if it is controlled by the wrong people, the people could provide the bad motives. 5. It seems like, however far we are from the singularity, we are getting closer. Our computers continue to do more intelligent things. 6. We don't yet have a good way to control AGI if we create it. In short, it seems like we are approaching something very dangerous and we don't know what to do when we get there. Seems like a literal problem to me.
  - athrowaway3z 6y ago
    >No known reason why AGI can't exist >it seems like we are approaching Sure on 1. But there is a jump in complexity required to do what a brain does. I'm not sure what the actual numbers are, but complexity doesn't scale linearly when everything is connected. We only have a single successful example of a super organism growing exponentially in its complexity. ( for now ). That took evolution millions of years from the first eye's doing image recognition distributed over untold number of brains. Us playing AI gods will surely speed it up, but not before something like global warming floods my floors. And where those kind of issues require action now, the issue of AI can be postponed at any moment by a power outage.

8 points by ZephyrBlu 6y ago

It is possible to build more general purpose AI without creating digital life that will compete with us. But people will stupidly go ahead and create digital emulations of humans with human-like desires and instincts but superhuman capabilities. When they do that, un-augmented humans will not be in control over the planet. The AIs will be. Maybe the people with Neuralink's or whatever can hope to be their friends. Or favored pets. But there is no way we can control superintelligence unless we explicitly design it to be nothing like people or other animals. It can't have any true autonomy or instincts, open-ended action, or many other capabilities and characteristics of creatures. If we are careful we may be able to stay in control of the planet for another 40 years, 100 years, maybe even 200 if we are lucky. But I think eventually someone will build digital superintelligent creatures that are (human) animal-like and completely autonomous out of a type of moral imperative that puts the evolution of intelligent life over the supremacy of humanity. I hope they wait until after my lifetime though.

deleted 6y ago

[deleted]

aabhay 6y ago

The AI control problem is a sci-fi narrative that is less instructive as a future speculation, and more instructive as an allegory for the present, in which a vanishingly small upper elite power class can construct a false reality for a destitute proletariat.

ALittleLight 6y ago
Why do you think that? My own intuition says that AI Control is a huge problem in the literal sense and not a metaphor at all. My reasoning: 1. No known reason why AGI can't exist. We know our general intelligence exists, so unless there is something magic about us, AGI should be possible. 2. Engineered things can surpass biological things on specific dimensions. A jet flies faster than a bird, a crane lifts more than an elephant, car faster than cheetah, etc. If the same concept holds for intelligence, we could build something much smarter than us. Put another way, we don't know how far above us the limit of intelligence is, so there is some unknown space into which our new intelligence may fall. It could be much smarter than us. 3. Recursive self-improvement, digital copies, horizontal scaling, never getting tired, faster physical limits on computation, perfect memory - AGI may have lots of advantages over us. 4. Intelligence more powerful than our own could do anything, including taking over the world, exterminating humanity, creating a virtual hell and trapping everyone in it. The AGI may not have a motive to do so, but if it is controlled by the wrong people, the people could provide the bad motives. 5. It seems like, however far we are from the singularity, we are getting closer. Our computers continue to do more intelligent things. 6. We don't yet have a good way to control AGI if we create it. In short, it seems like we are approaching something very dangerous and we don't know what to do when we get there. Seems like a literal problem to me.
- athrowaway3z 6y ago
  >No known reason why AGI can't exist >it seems like we are approaching Sure on 1. But there is a jump in complexity required to do what a brain does. I'm not sure what the actual numbers are, but complexity doesn't scale linearly when everything is connected. We only have a single successful example of a super organism growing exponentially in its complexity. ( for now ). That took evolution millions of years from the first eye's doing image recognition distributed over untold number of brains. Us playing AI gods will surely speed it up, but not before something like global warming floods my floors. And where those kind of issues require action now, the issue of AI can be postponed at any moment by a power outage.