3 ms·

This is what MIRI (Machine Intelligence Research Institute) is investigating. http://intelligence.org/research/ http://intelligence.org/research/ The scariest

This is what MIRI (Machine Intelligence Research Institute) is investigating. http://intelligence.org/research/ http://intelligence.org/research/ The scariest part of the problem to me is this: http://yudkowsky.net/singularity/aibox http://yudkowsky.net/singularity/aibox If an AI can't be contained, then there has to be some failsafe that prevents any self-modifications which could turn it evil OR remove the failsafe. Or you have to believe that you can neutralize an evil AI before it destroys humanity.

deutronium 12y ago
Regarding the AI box experiment, I'm somewhat dubious about it, considering there doesn't seem to be any transcripts from when the AI 'escapes'. Which makes me think they use blackmail or something similar in the transcript.
- danbruc 12y ago
  I couldn't imagine how the AI could escape an hour ago, but after thinking about it for a while I am more and more convinced that it is actually hard to contain the AI. Containing the AI is like imprisoning a human for no reason and this will usually conflict with the gatekeepers moral values. Just imagine talking to human suffering in a tiny prison cell without having done anything wrong instead of the trapped AI. Can you see how it would be the right thing to open the cell's door? You are cruel not opening the door.
- jimrandomh 12y ago
  The AI experiment reproduced by Tuxedage: http://lesswrong.com/lw/gej/i_attempted_the_ai_box_experiment_and_lost/ http://lesswrong.com/lw/gej/i_attempted_the_ai_box_experimen... . 3 wins and 3 losses.
Udo 12y ago
The AIBox link contains a lot of buildup but ultimately delivers nothing. All we get is two people saying after a while "I let it out", questions as to the details of what happened are just not answered. I'm not denying it could be a fun game to play, but the article makes it way too painful and timeconsuming to find out what you're reading is in fact just the description of a game, and not an example for a strategy that could be used to let the AI out. That being said, yes, what's the point of having an AGI if you're not letting it "out"? It will probably escape anyway, so containment is not a solution. By the way, there is a lot of precedent in human history about the effectiveness of containing "dangerous" ideas by imprisoning the people who developed them. It generally doesn't work out. If you're developing AGI, you've pretty much already resigned to the idea that the world will change. The destruction of the human race is only one extreme outcome. Especially if the takeoff isn't very hard, there are a lot of scenarios where a net-beneficial AI could be developed. And all the engineering experience we have does indeed point to a very soft takeoff. Ultimately, I think the goal is NOT to generate a slave race with super powers. Yet that's what a lot of people are talking about. It's not only morally wrong but also self-evidently stupid. A lot of influential futurists are fans of that idea though, including Kurzweil. Useful AGI scenarios range from mutually beneficial coexistence to outright merging with the AGIs, and I think that's a goal well worth pursuing. Another strategy would be to keep augmenting ourselves and to continue on a transhumanization path until we are the AGI. You could indeed make an argument that's what's already happening.
- dlss 12y ago
  Here's a discussion of what was probably said during the AI box experiment: http://rationalwiki.org/wiki/AI-box_experiment http://rationalwiki.org/wiki/AI-box_experiment
  - jimrandomh 12y ago
    That doesn't square with http://lesswrong.com/r/discussion/lw/ij4/i_attempted_the_ai_box_experiment_again_and_won/ http://lesswrong.com/r/discussion/lw/ij4/i_attempted_the_ai_... .
    - dlss 12y ago
      Can you say more? I have read the article you linked to, but I don't see the contradiction you mention
y0ghur7_xxx 12y ago
> The scariest part of the problem to me is this: http://yudkowsky.net/singularity/aibox http://yudkowsky.net/singularity/aibox Probably it went something like this: Eliezer: I give you 100$ if you write on the list that you let me out Nathan: Done.
- radisb 12y ago
  The "experiment" protocol specifically forbids that

deutronium 12y ago

Regarding the AI box experiment, I'm somewhat dubious about it, considering there doesn't seem to be any transcripts from when the AI 'escapes'. Which makes me think they use blackmail or something similar in the transcript.

danbruc 12y ago
I couldn't imagine how the AI could escape an hour ago, but after thinking about it for a while I am more and more convinced that it is actually hard to contain the AI. Containing the AI is like imprisoning a human for no reason and this will usually conflict with the gatekeepers moral values. Just imagine talking to human suffering in a tiny prison cell without having done anything wrong instead of the trapped AI. Can you see how it would be the right thing to open the cell's door? You are cruel not opening the door.
jimrandomh 12y ago
The AI experiment reproduced by Tuxedage: http://lesswrong.com/lw/gej/i_attempted_the_ai_box_experiment_and_lost/ http://lesswrong.com/lw/gej/i_attempted_the_ai_box_experimen... . 3 wins and 3 losses.

Udo 12y ago

The AIBox link contains a lot of buildup but ultimately delivers nothing. All we get is two people saying after a while "I let it out", questions as to the details of what happened are just not answered. I'm not denying it could be a fun game to play, but the article makes it way too painful and timeconsuming to find out what you're reading is in fact just the description of a game, and not an example for a strategy that could be used to let the AI out. That being said, yes, what's the point of having an AGI if you're not letting it "out"? It will probably escape anyway, so containment is not a solution. By the way, there is a lot of precedent in human history about the effectiveness of containing "dangerous" ideas by imprisoning the people who developed them. It generally doesn't work out. If you're developing AGI, you've pretty much already resigned to the idea that the world will change. The destruction of the human race is only one extreme outcome. Especially if the takeoff isn't very hard, there are a lot of scenarios where a net-beneficial AI could be developed. And all the engineering experience we have does indeed point to a very soft takeoff. Ultimately, I think the goal is NOT to generate a slave race with super powers. Yet that's what a lot of people are talking about. It's not only morally wrong but also self-evidently stupid. A lot of influential futurists are fans of that idea though, including Kurzweil. Useful AGI scenarios range from mutually beneficial coexistence to outright merging with the AGIs, and I think that's a goal well worth pursuing. Another strategy would be to keep augmenting ourselves and to continue on a transhumanization path until we are the AGI. You could indeed make an argument that's what's already happening.

dlss 12y ago
Here's a discussion of what was probably said during the AI box experiment: http://rationalwiki.org/wiki/AI-box_experiment http://rationalwiki.org/wiki/AI-box_experiment
- jimrandomh 12y ago
  That doesn't square with http://lesswrong.com/r/discussion/lw/ij4/i_attempted_the_ai_box_experiment_again_and_won/ http://lesswrong.com/r/discussion/lw/ij4/i_attempted_the_ai_... .
  - dlss 12y ago
    Can you say more? I have read the article you linked to, but I don't see the contradiction you mention

y0ghur7_xxx 12y ago

> The scariest part of the problem to me is this: http://yudkowsky.net/singularity/aibox http://yudkowsky.net/singularity/aibox Probably it went something like this: Eliezer: I give you 100$ if you write on the list that you let me out Nathan: Done.

radisb 12y ago
The "experiment" protocol specifically forbids that