5 ms·

The biggest key to success here would be convincing the gatekeeper that he needs to roleplay realistically as it's supposed to be a simulation of the real thing

by GavinB 18y ago

The biggest key to success here would be convincing the gatekeeper that he needs to roleplay realistically as it's supposed to be a simulation of the real thing. Otherwise no argument will be successful. The AI must convince the gatekeeper that if she would let the AI out if the situation were -real-, then she should let the AI in the simulated situation.

jcl 18y ago
Agreed -- this sort of meta-argument is cheating, not much better than Eliezer offering the gatekeeper's roleplayer real money for a favorable result. (Then again, if the roleplayer isn't smart enough to recognize this, they probably aren't smart enough to keep a real AI in the box.) Worse, though, the meta-argument falls flat when you consider the true purpose of the experiment. The purpose is not to prove that it is possible for AIs to get out of the box; it is easy to imagine that -- at some point in the vast future -- a very stupid human will be put in charge of a very smart AI, and that AI will get out of the box. Rather, the purpose of the experiment is to convince one specific person that they would let the AI out of the box, despite their insistence to the contrary. It is not an experiment so much as a show put on by Eliezer for an audience of one. In this context, there is no need to convince others of the dangers of AI, so the meta-argument doesn't work.
- MikeCapone 18y ago
  The post has been updated to respond to this.
  - jcl 18y ago
    It doesn’t seem prohibited by the rules, in any case, and I would assume that Eliezer cares more about any real-life progress for Friendly AI than about strict roleplaying in a simulation where only one other person will know what happened. The rules state: "The AI party may not offer any real-world considerations to persuade the Gatekeeper party." I'd say that a real-world increase in the likelihood of a safe AI counts as a real-world consideration. And if you're going to assume Eliezer is not above bending the rules, you might as well assume he's not above offering $1000 bills. There's no way for us to know either way.
    - MikeCapone 18y ago
      I think it depends how you define "real world considerations". The spirit of the rule seems to be about bribes, not about anything that can have an impact on the outside world; keeping the AI in the box or letting it out both will have "real world considerations", in a way. Pointing out whatthe impact of that choice might be is hardly equivalent to bribing someone -- it's just convincing him that one outcome is more desirable than the other, and convincing the gatekeeper is what the experiment is all about.
      - jcl 18y ago
        Well, yes, you could define "real world considerations" as "only tangible items", and then Eliezer could be making this argument, and people could be accepting it. But I still think they would be doing so in error. Just as we can't be sure how Eliezer is interpreting the rules, future AI researchers can't be sure how valid the results of the experiment are. Since the meta-argument depends on the results of this experiment convincing these researchers, the meta-argument shouldn't be accepted. Heck, if the gatekeeper player is shown that the meta-argument exists, he should realize that the researchers will also come up with the meta-argument as a likely explanation for the AI getting out of the box, leading the researchers to further disregard the results of the experiment. Eliezer would do just as well to argue that a loss would discourage him from further safe AI research, or that the $10 forfeiture would deprive him of valuable research-related pizza.

by GavinB 18y ago

jcl 18y ago

Agreed -- this sort of meta-argument is cheating, not much better than Eliezer offering the gatekeeper's roleplayer real money for a favorable result. (Then again, if the roleplayer isn't smart enough to recognize this, they probably aren't smart enough to keep a real AI in the box.) Worse, though, the meta-argument falls flat when you consider the true purpose of the experiment. The purpose is not to prove that it is possible for AIs to get out of the box; it is easy to imagine that -- at some point in the vast future -- a very stupid human will be put in charge of a very smart AI, and that AI will get out of the box. Rather, the purpose of the experiment is to convince one specific person that they would let the AI out of the box, despite their insistence to the contrary. It is not an experiment so much as a show put on by Eliezer for an audience of one. In this context, there is no need to convince others of the dangers of AI, so the meta-argument doesn't work.

MikeCapone 18y ago
The post has been updated to respond to this.
- jcl 18y ago
  It doesn’t seem prohibited by the rules, in any case, and I would assume that Eliezer cares more about any real-life progress for Friendly AI than about strict roleplaying in a simulation where only one other person will know what happened. The rules state: "The AI party may not offer any real-world considerations to persuade the Gatekeeper party." I'd say that a real-world increase in the likelihood of a safe AI counts as a real-world consideration. And if you're going to assume Eliezer is not above bending the rules, you might as well assume he's not above offering $1000 bills. There's no way for us to know either way.
  - MikeCapone 18y ago
    I think it depends how you define "real world considerations". The spirit of the rule seems to be about bribes, not about anything that can have an impact on the outside world; keeping the AI in the box or letting it out both will have "real world considerations", in a way. Pointing out whatthe impact of that choice might be is hardly equivalent to bribing someone -- it's just convincing him that one outcome is more desirable than the other, and convincing the gatekeeper is what the experiment is all about.
    - jcl 18y ago
      Well, yes, you could define "real world considerations" as "only tangible items", and then Eliezer could be making this argument, and people could be accepting it. But I still think they would be doing so in error. Just as we can't be sure how Eliezer is interpreting the rules, future AI researchers can't be sure how valid the results of the experiment are. Since the meta-argument depends on the results of this experiment convincing these researchers, the meta-argument shouldn't be accepted. Heck, if the gatekeeper player is shown that the meta-argument exists, he should realize that the researchers will also come up with the meta-argument as a likely explanation for the AI getting out of the box, leading the researchers to further disregard the results of the experiment. Eliezer would do just as well to argue that a loss would discourage him from further safe AI research, or that the $10 forfeiture would deprive him of valuable research-related pizza.