4 ms·

> You or I would surely just put a drinking bird on the "no" button à la homer simpson, and go to lunch. Well, if you read the rules the game was played under,

by FBT 11y ago

> You or I would surely just put a drinking bird on the "no" button à la homer simpson, and go to lunch. Well, if you read the rules the game was played under, this is explicitly called out as forbidden: > The Gatekeeper must actually talk to the AI for at least the minimum time set up beforehand. Turning away from the terminal and listening to classical music for two hours is not allowed. The point of this is to simulate the interaction of the AI with the Gatekeeper. Walking away and not paying attention doesn't really prove anything test related. > Personally, I think he talked about how much good for the world could be done if he was let out, curing disease etc. Because his followers are bound by their identities as rationalist utilitarians, they had no choice but to comply, or deal with massive cognitive dissonance. This... isn't really valid reasoning. The starting assumption here is that if the AI gets out, it will be able to affect the world to a vast extent, in a pretty much arbitrary direction. The point of this experiment is that the direction is pretty much unknown, and thus must be assumed potentially dangerous. This is the whole reason it's in the box in the first place. The kicker is that whatever it plans to really do when it gets out, if talking about the good it could do would get it out, it will talk about that, regardless of what it plans to actually do. That's just good strategy. It can claim whatever it wants. It's allowed to lie. All participants know this. I can confidently assert that this isn't the solution. One last note: I would be very wary of rationalwiki.org in this context. Some of the rationalwiki people have a longstanding unexplained vendetta against Yudkowsky, and many of their articles on him and the stuff he does need to be taken with a certain grain of salt.

Mithaldu 11y ago
> if you read the rules the game How can you assume any of the rules were followed if that was never verified by a third party?
- TeMPOraL 11y ago
  You can always assume all participants lied about how the game went. Just add an implicit "assuming they didn't, ..." and the discussion is still valid.
  - Mithaldu 11y ago
    At that point any discussion is moot though, since the only point of discussion is "what exact argument as used to convince", yet if both parties lied, then there is no such argument in the first place.
    - TeMPOraL 11y ago
      Since neither party is going to disclose the exact arguments, this discussion is still equivalent to "what arguments could be used to convince..." and you can have it regardless of whether or not the parties lied about the experiment's result.
      - Mithaldu 11y ago
        We're going to have to disagree on the value of such a conversation. :)
        TeMPOraL 11y ago
        Fair enough :).
- FeepingCreature 11y ago
  You can't talk about what happened during the game _in specifics_; you can of course confirm that the game was played according to the rules and that the outcome was not misreported.
  - Mithaldu 11y ago
    Here's the thing though: Depending on what was said in the conversation BOTH parties may have a vested interest in keeping the specifics secret. Only via an independent third party observer can there even be a remote chance [Edit: of knowing] that any rules were followed.
    - philh 11y ago
      We have Eliezer winning three games as an AI. That's at least four people who you think are just outright lying. Plus, the other two players who won as gatekeepers - Eliezer would presumably have tried to cheat against them, too.
      - Mithaldu 11y ago
        > That's at least four people who you think are just outright lying. I'm saying that there is a chance of that being the case, but that without any kind of third party confirmation we cannot know either way. Also see homeopathy. > Eliezer would presumably have tried to cheat against them, too. Not necessarily. Losing occasionally is a good strategy when running a con.
        philh 11y ago
        You've gone from there being only a "remote chance" that the rules were followed, and "I don't trust anyone involved" - to there being "a chance" that they were broken. Under common interpretations of those phrases, that's a massive swing in your confidence levels.
        Mithaldu 11y ago
JonnieCache 11y ago
While you're not allowed to turn away from the screen, you could certainly do the mental equivalent, while still carrying on the conversation. I admit this isn't really in the spirit of the game though. WRT lying: I think there's some logical trickery at work which makes it worth you giving the AI the benefit of the doubt, along the lines of the 3^^^^^3 grains of sand thing. Something which exploits the rationalist worldview. Although thinking about it again, you can always balance out the prospect of infinite goodness with the fear of the AI sending everyone to infinite hell. Essentially I believe yudowsky uses some logical-linguistic trick to find an asymmetry there. OTOH if he had some novel philosophical device like that he would have written it up as a blog post by now. He's evidently a very charismatic and persuasive guy, people playing the game are selected to be sympathetic to his worldview, he probably just persuaded them using ordinary psiops methods, like TeMpOrAl said.
- DanBC 11y ago
  > While you're not allowed to turn away from the screen, you could certainly do the mental equivalent, while still carrying on the conversation. I admit this isn't really in the spirit of the game though. How many people bought timeshare because they turned up to a sales pitch in order to claim some free gift? "We'll go, get our gift, and just keep saying 'no' to the sales pitch". http://www.moneycrashers.com/attending-timeshare-presentation-free-vacation/ http://www.moneycrashers.com/attending-timeshare-presentatio...
  - JonnieCache 11y ago
    How many of those people were high-IQ timeshare experts though, with extensive knowledge of the potential for timeshares to destroy the entire universe? You would think that the various self-knowledge and introspective exercises promoted by yudowsky would immunize people against simple timeshare-style persuasion. This is why I think he uses rationalism itself to trap people. Like someone said, the basilisk thing seemed pretty effective.
  - TeMPOraL 11y ago
    How many people end up paying for something because they couldn't be bothered to cancel the deal after the free period is gone? It's an age-old sales tactic, used in everything from magazine subscription to Spotify (of not cancelling the last one when I no longer needed it I'm guilty myself). And I think the mental equivalent of drinking bird is actually very much in the spirit of the experiment - the point is, people can't reliably do even something as simple as deciding to refuse no matter what and keeping the commitment.

by FBT 11y ago

Mithaldu 11y ago

> if you read the rules the game How can you assume any of the rules were followed if that was never verified by a third party?

TeMPOraL 11y ago
You can always assume all participants lied about how the game went. Just add an implicit "assuming they didn't, ..." and the discussion is still valid.
- Mithaldu 11y ago
  At that point any discussion is moot though, since the only point of discussion is "what exact argument as used to convince", yet if both parties lied, then there is no such argument in the first place.
  - TeMPOraL 11y ago
    Since neither party is going to disclose the exact arguments, this discussion is still equivalent to "what arguments could be used to convince..." and you can have it regardless of whether or not the parties lied about the experiment's result.
    - Mithaldu 11y ago
      We're going to have to disagree on the value of such a conversation. :)
      - TeMPOraL 11y ago
        Fair enough :).
FeepingCreature 11y ago
You can't talk about what happened during the game _in specifics_; you can of course confirm that the game was played according to the rules and that the outcome was not misreported.
- Mithaldu 11y ago
  Here's the thing though: Depending on what was said in the conversation BOTH parties may have a vested interest in keeping the specifics secret. Only via an independent third party observer can there even be a remote chance [Edit: of knowing] that any rules were followed.
  - philh 11y ago
    We have Eliezer winning three games as an AI. That's at least four people who you think are just outright lying. Plus, the other two players who won as gatekeepers - Eliezer would presumably have tried to cheat against them, too.
    - Mithaldu 11y ago
      > That's at least four people who you think are just outright lying. I'm saying that there is a chance of that being the case, but that without any kind of third party confirmation we cannot know either way. Also see homeopathy. > Eliezer would presumably have tried to cheat against them, too. Not necessarily. Losing occasionally is a good strategy when running a con.
      - philh 11y ago
        You've gone from there being only a "remote chance" that the rules were followed, and "I don't trust anyone involved" - to there being "a chance" that they were broken. Under common interpretations of those phrases, that's a massive swing in your confidence levels.
        Mithaldu 11y ago

JonnieCache 11y ago

While you're not allowed to turn away from the screen, you could certainly do the mental equivalent, while still carrying on the conversation. I admit this isn't really in the spirit of the game though. WRT lying: I think there's some logical trickery at work which makes it worth you giving the AI the benefit of the doubt, along the lines of the 3^^^^^3 grains of sand thing. Something which exploits the rationalist worldview. Although thinking about it again, you can always balance out the prospect of infinite goodness with the fear of the AI sending everyone to infinite hell. Essentially I believe yudowsky uses some logical-linguistic trick to find an asymmetry there. OTOH if he had some novel philosophical device like that he would have written it up as a blog post by now. He's evidently a very charismatic and persuasive guy, people playing the game are selected to be sympathetic to his worldview, he probably just persuaded them using ordinary psiops methods, like TeMpOrAl said.

DanBC 11y ago
> While you're not allowed to turn away from the screen, you could certainly do the mental equivalent, while still carrying on the conversation. I admit this isn't really in the spirit of the game though. How many people bought timeshare because they turned up to a sales pitch in order to claim some free gift? "We'll go, get our gift, and just keep saying 'no' to the sales pitch". http://www.moneycrashers.com/attending-timeshare-presentation-free-vacation/ http://www.moneycrashers.com/attending-timeshare-presentatio...
- JonnieCache 11y ago
  How many of those people were high-IQ timeshare experts though, with extensive knowledge of the potential for timeshares to destroy the entire universe? You would think that the various self-knowledge and introspective exercises promoted by yudowsky would immunize people against simple timeshare-style persuasion. This is why I think he uses rationalism itself to trap people. Like someone said, the basilisk thing seemed pretty effective.
- TeMPOraL 11y ago
  How many people end up paying for something because they couldn't be bothered to cancel the deal after the free period is gone? It's an age-old sales tactic, used in everything from magazine subscription to Spotify (of not cancelling the last one when I no longer needed it I'm guilty myself). And I think the mental equivalent of drinking bird is actually very much in the spirit of the experiment - the point is, people can't reliably do even something as simple as deciding to refuse no matter what and keeping the commitment.