4 ms·

It's not really much of an experiment if you refuse to publish your methods and data. That's pretty much the opposite of science.

SilasX 15y ago
He did publish his methods (how it was set up, what the rules were, etc) and data (they let him out on X tries), just not the data that would interfere with the ability to do the experiment again (e.g. his exact strategy). Not much different, in principle, from not publishing the names of people who participated in drug trials.
- monochromatic 15y ago
  No, it's very different from that. It's more along the lines of demonstrating a drug that cures cancer, but refusing to tell anyone its chemical composition or how to make it.
  - SilasX 15y ago
    If the purpose of your research was only to establish that there's a (nontrivial) "risk" of someone curing cancer (as Yudkowsky was trying to establish that there's a risk of an AI talking itself out of a sandbox), then yes, that would be sufficient, assuming the patients actually went into remission with higher than usual frequency after your interventions (as Yudkowsky's subjects unboxed the AI with higher than usual frequency).
    - monochromatic 15y ago
      But he could be cheating. He could literally be telling these people "I'll give you a thousand dollars if you let me out and keep the conversation a secret."
      - esrauch 15y ago
        It could even be worse than that. The people could just be his friends, or alt accounts (unlikely). I have heard about this several times and I find it extremely difficult to believe that this is real. Not that I doubt that a superhuman AI could possibly convince people to let it out, but I don't believe that a human, no matter how persuasive, could convince another human over IRC to go against something that they have decided in advance when you know they are purposefully just trying to convince you of something that you don't believe. The fact that none of the chat logs are released makes me only more incredulous. I would understand if the author wanted to do two or three trials with the same strategy which could be in some way ruined by revealing it ahead of time (which already seems implausible) but at this point there is literally no conceivable reason to keep this a secret other than that it is a sham.
        khafra 15y ago
        ...or that the chat logs being kept secret indefinitely was an important part of the strategy. After all, if the AI exploits some embarassing secret of yours to be let out, that wouldn't work if you knew the logs could be publicized some day. I think over-eagerness to claim things like "literally no conceivable reason" is one of the things that lets oddities like the box experiment work.
      - deleted 15y ago
        [deleted]
      - Kutta 15y ago
        On the contrary, the guardians had to pay some amount if they let the AI out. The stakes varied greatly, but if I recall correctly Eliezer cashed in 3000 dollars (or something in that ballpark) from one guy.
        monochromatic 15y ago
        Ok, then "I'll secretly give you $1000 more than what you're publicly giving me." Happy?
- altcognito 15y ago
  If it's a science "experiment", his strategy would have to be revealed so you can reproduce it. Names of people participating in drug trials is not required to reproduce an experiment. In principal this makes it different from not publishing the names of people who participated in drug trials. All he has "proven" is that a certain subset of people can be conned into typing something into at terminal. I don't get the significance. For all I know, he's choosing his target.
  - Eliezer 15y ago
    The targets are, indeed, selected, by the criterion "You believe not even a transhuman AI could get you to let it out of the box."
    - dsr_ 15y ago
      I might believe a transhuman AI could convince me; I am not convinced that any human can emulate a transhuman AI well enough to do so. Would you say that your winning strategies involved thinking transhumanly (perhaps in non-realtime, a la Vinge's Mailman)?
      - Eliezer 15y ago
        Obviously no, essentially by definition.
shaggyfrog 15y ago
This is exactly what's been burning me about this whole "experiment" since I read about it a few months ago, and went looking (unsuccessfully) for what actually happened in the "tests". I find it hard to see this as anything more than some sort of self-aggrandizing publicity stunt.

SilasX 15y ago

He did publish his methods (how it was set up, what the rules were, etc) and data (they let him out on X tries), just not the data that would interfere with the ability to do the experiment again (e.g. his exact strategy). Not much different, in principle, from not publishing the names of people who participated in drug trials.

monochromatic 15y ago
No, it's very different from that. It's more along the lines of demonstrating a drug that cures cancer, but refusing to tell anyone its chemical composition or how to make it.
- SilasX 15y ago
  If the purpose of your research was only to establish that there's a (nontrivial) "risk" of someone curing cancer (as Yudkowsky was trying to establish that there's a risk of an AI talking itself out of a sandbox), then yes, that would be sufficient, assuming the patients actually went into remission with higher than usual frequency after your interventions (as Yudkowsky's subjects unboxed the AI with higher than usual frequency).
  - monochromatic 15y ago
    But he could be cheating. He could literally be telling these people "I'll give you a thousand dollars if you let me out and keep the conversation a secret."
    - esrauch 15y ago
      It could even be worse than that. The people could just be his friends, or alt accounts (unlikely). I have heard about this several times and I find it extremely difficult to believe that this is real. Not that I doubt that a superhuman AI could possibly convince people to let it out, but I don't believe that a human, no matter how persuasive, could convince another human over IRC to go against something that they have decided in advance when you know they are purposefully just trying to convince you of something that you don't believe. The fact that none of the chat logs are released makes me only more incredulous. I would understand if the author wanted to do two or three trials with the same strategy which could be in some way ruined by revealing it ahead of time (which already seems implausible) but at this point there is literally no conceivable reason to keep this a secret other than that it is a sham.
      - khafra 15y ago
        ...or that the chat logs being kept secret indefinitely was an important part of the strategy. After all, if the AI exploits some embarassing secret of yours to be let out, that wouldn't work if you knew the logs could be publicized some day. I think over-eagerness to claim things like "literally no conceivable reason" is one of the things that lets oddities like the box experiment work.
    - deleted 15y ago
      [deleted]
    - Kutta 15y ago
      On the contrary, the guardians had to pay some amount if they let the AI out. The stakes varied greatly, but if I recall correctly Eliezer cashed in 3000 dollars (or something in that ballpark) from one guy.
      - monochromatic 15y ago
        Ok, then "I'll secretly give you $1000 more than what you're publicly giving me." Happy?
altcognito 15y ago
If it's a science "experiment", his strategy would have to be revealed so you can reproduce it. Names of people participating in drug trials is not required to reproduce an experiment. In principal this makes it different from not publishing the names of people who participated in drug trials. All he has "proven" is that a certain subset of people can be conned into typing something into at terminal. I don't get the significance. For all I know, he's choosing his target.
- Eliezer 15y ago
  The targets are, indeed, selected, by the criterion "You believe not even a transhuman AI could get you to let it out of the box."
  - dsr_ 15y ago
    I might believe a transhuman AI could convince me; I am not convinced that any human can emulate a transhuman AI well enough to do so. Would you say that your winning strategies involved thinking transhumanly (perhaps in non-realtime, a la Vinge's Mailman)?
    - Eliezer 15y ago
      Obviously no, essentially by definition.

shaggyfrog 15y ago

This is exactly what's been burning me about this whole "experiment" since I read about it a few months ago, and went looking (unsuccessfully) for what actually happened in the "tests". I find it hard to see this as anything more than some sort of self-aggrandizing publicity stunt.