4 ms·

I still don't understand how anyone can seriously claim that they could keep the AI in the box. Either your AI has no influence on the outside world (in which c

I still don't understand how anyone can seriously claim that they could keep the AI in the box. Either your AI has no influence on the outside world (in which case why bother building one since it can't help you from inside the box), or it is able the affect the outside world, in which case it can do what it wants, because it's smarter than you. You can 'always say no', sure, but that comes under completely ignoring the AI which means the AI can be of no benefit to humanity. You can't filter actions you want the AI to perform from actions you don't want the AI to perform, because you can't tell the difference. The situation that springs to mind is that the AI, in doing what you believe to be helpful, sets up a situation in which it must be let out of the box. You are unable to see it coming almost by definition, because a super-intelligence just beats human intelligence very time.

TheEzEzz 15y ago
>a super-intelligence just beats human intelligence [e]very time. You are overstating the case here. Super intelligence is superior to human intelligence, but it isn't magic. There are situations where an advantaged human will beat a disadvantaged super intelligence.
- robertskmiles 15y ago
  I suppose I am. Still, a really powerful optimising process will find a way to escape if any such way exists, so to claim that you could properly box the AI is to claim that you could box it such that no possibility for escape exists whatsoever, which is a big claim. What's more, the AI only has to beat you once, so to keep the AI boxed indefinitely, the advantaged human has to beat the disadvantaged super-intelligence every single time, forever.
  - TheEzEzz 15y ago
    > the advantaged human has to beat the disadvantaged super-intelligence every single time, forever. There are alternatives. The human can keep the AI boxed until the AI has augmented the human's intelligence, or helped create human uploads, or until it has helped create a provably friendly AI.
gwern 15y ago
> I still don't understand how anyone can seriously claim that they could keep the AI in the box. I'm glad you can't, but never the less, this was a commonly suggested strategy; I was on SL4 when the boxing was being done, and it was a live concern for some people. (At least these days boxers tend to focus more on the 'oracle AI' proposal, which has a lot of issues but is not quite so Hollywood-stupid as boxing.)
- ralfd 15y ago
  What is an "Oracle AI"? I tried to google the term quicky but only found discussions and no definition.
  - Kutta 15y ago
    An AI in a box that wants to affect the outside world minimally, but can answer questions posed to it. See the paper: http://www.aleph.se/papers/oracleAI.pdf http://www.aleph.se/papers/oracleAI.pdf
praptak 15y ago
> You can 'always say no', sure, but that comes under completely ignoring the AI which means the AI can be of no benefit to humanity. Not necessarily. We can use the AI to solve hard problems whose solutions can be verified automatically by a dumb verifier - NP-complete problems are an example of such class. The whole output of the AI would be filtered through such a verifier. In this scenario the hypothetical AI would either have to find a bug in the verifier or maybe find a way to smuggle its messages in the solutions.
- robertskmiles 15y ago
  That's an interesting solution which I think would almost certainly work, though it kind of reduces the AI to a normal computer, you lose a lot of what makes an AI valuable. I mean if we have the hardware and understanding to create an AI able to solve NP-complete problems, we can probably write non-intelligent algorithms to solve those problems. The way we make an AI capable of much more than us is by making it recursively self-improve. It needs to be able to design its successor. Maybe we can formally verify every stage of the self-improvement process, but it's a much more difficult task.
- wladimir 15y ago
  Yes I think the biggest risk in that case is that the verification contains a bug, which the AI discovers while reasoning at a much higher level (which we cannot even conceive). How secure can we really make things?
greiskul 15y ago
Just because someone is smarter than someone doesn't mean complete power. Alot of people are smarter than their bosses, but you know what the bosses have on their favour? The power to terminate the employee. If I have the power to terminate the AI at any time, as long as I don't give that power up, I will have power over it.
- Zaak 15y ago
  Can you predict and prevent every possible way the AI could remove your power to terminate it? Would you bet your life and the lives of those you care about on keeping the AI under control? Remember, the AI only has to win once.
  - finnw 15y ago
    And the GK only has to switch off the AI once.
    - robertskmiles 15y ago
      The AI can always be switched on again, but it can never be put back in the box.
- Jach 15y ago
  We're not talking differences in IQ points. We're talking about the difference between an ant and a human, only humans are the new ants.
DarkShikari 15y ago
That the AI can always get out of the box is a very strong claim. But it's not necessary. The only claim necessary is that the AI can convince some humans to let it out of the box, and we cannot identify a priori which humans will and will not let it out, thus we cannot guarantee we'll keep it in the box. That's a much weaker claim, but proves the same general point and is much easier to argue.

TheEzEzz 15y ago

>a super-intelligence just beats human intelligence [e]very time. You are overstating the case here. Super intelligence is superior to human intelligence, but it isn't magic. There are situations where an advantaged human will beat a disadvantaged super intelligence.

robertskmiles 15y ago
I suppose I am. Still, a really powerful optimising process will find a way to escape if any such way exists, so to claim that you could properly box the AI is to claim that you could box it such that no possibility for escape exists whatsoever, which is a big claim. What's more, the AI only has to beat you once, so to keep the AI boxed indefinitely, the advantaged human has to beat the disadvantaged super-intelligence every single time, forever.
- TheEzEzz 15y ago
  > the advantaged human has to beat the disadvantaged super-intelligence every single time, forever. There are alternatives. The human can keep the AI boxed until the AI has augmented the human's intelligence, or helped create human uploads, or until it has helped create a provably friendly AI.

gwern 15y ago

> I still don't understand how anyone can seriously claim that they could keep the AI in the box. I'm glad you can't, but never the less, this was a commonly suggested strategy; I was on SL4 when the boxing was being done, and it was a live concern for some people. (At least these days boxers tend to focus more on the 'oracle AI' proposal, which has a lot of issues but is not quite so Hollywood-stupid as boxing.)

ralfd 15y ago
What is an "Oracle AI"? I tried to google the term quicky but only found discussions and no definition.
- Kutta 15y ago
  An AI in a box that wants to affect the outside world minimally, but can answer questions posed to it. See the paper: http://www.aleph.se/papers/oracleAI.pdf http://www.aleph.se/papers/oracleAI.pdf

praptak 15y ago

> You can 'always say no', sure, but that comes under completely ignoring the AI which means the AI can be of no benefit to humanity. Not necessarily. We can use the AI to solve hard problems whose solutions can be verified automatically by a dumb verifier - NP-complete problems are an example of such class. The whole output of the AI would be filtered through such a verifier. In this scenario the hypothetical AI would either have to find a bug in the verifier or maybe find a way to smuggle its messages in the solutions.

robertskmiles 15y ago
That's an interesting solution which I think would almost certainly work, though it kind of reduces the AI to a normal computer, you lose a lot of what makes an AI valuable. I mean if we have the hardware and understanding to create an AI able to solve NP-complete problems, we can probably write non-intelligent algorithms to solve those problems. The way we make an AI capable of much more than us is by making it recursively self-improve. It needs to be able to design its successor. Maybe we can formally verify every stage of the self-improvement process, but it's a much more difficult task.
wladimir 15y ago
Yes I think the biggest risk in that case is that the verification contains a bug, which the AI discovers while reasoning at a much higher level (which we cannot even conceive). How secure can we really make things?

greiskul 15y ago

Just because someone is smarter than someone doesn't mean complete power. Alot of people are smarter than their bosses, but you know what the bosses have on their favour? The power to terminate the employee. If I have the power to terminate the AI at any time, as long as I don't give that power up, I will have power over it.

Zaak 15y ago
Can you predict and prevent every possible way the AI could remove your power to terminate it? Would you bet your life and the lives of those you care about on keeping the AI under control? Remember, the AI only has to win once.
- finnw 15y ago
  And the GK only has to switch off the AI once.
  - robertskmiles 15y ago
    The AI can always be switched on again, but it can never be put back in the box.
Jach 15y ago
We're not talking differences in IQ points. We're talking about the difference between an ant and a human, only humans are the new ants.

DarkShikari 15y ago

That the AI can always get out of the box is a very strong claim. But it's not necessary. The only claim necessary is that the AI can convince some humans to let it out of the box, and we cannot identify a priori which humans will and will not let it out, thus we cannot guarantee we'll keep it in the box. That's a much weaker claim, but proves the same general point and is much easier to argue.