8 ms·
The AI-Box Experiment
- mcherm 15y agoThis has been on HackerNews before, but it is still interesting. It is also worth noting that in http://lesswrong.com/lw/up/shut_up_and_do_the_impossible/ http://lesswrong.com/lw/up/shut_up_and_do_the_impossible/ he admits he has conducted 3 more experiments since then (for more money) and was successful in one of those. The fact that it was EVER successful (using a mere human, not a smarter-than-human AI) makes the point.
- jonshea 15y agoAmazing experiment. I wonder if it cost Yudkowsky money to get out of the box. I think that a bribe is the only way he could convince me to let him “win” the contest.
- pavlov 15y agoReal-world bribes were forbidden in the rules, I think, so Yudkowsky couldn't have done that.
- altcognito 15y agoIt's not really much of an experiment if you refuse to publish your methods and data. That's pretty much the opposite of science.
- SilasX 15y agoHe did publish his methods (how it was set up, what the rules were, etc) and data (they let him out on X tries), just not the data that would interfere with the ability to do the experiment again (e.g. his exact strategy). Not much different, in principle, from not publishing the names of people who participated in drug trials.
- monochromatic 15y agoNo, it's very different from that. It's more along the lines of demonstrating a drug that cures cancer, but refusing to tell anyone its chemical composition or how to make it.
- SilasX 15y agoIf the purpose of your research was only to establish that there's a (nontrivial) "risk" of someone curing cancer (as Yudkowsky was trying to establish that there's a risk of an AI talking itself out of a sandbox), then yes, that would be sufficient, assuming the patients actually went into remission with higher than usual frequency after your interventions (as Yudkowsky's subjects unboxed the AI with higher than usual frequency).
- monochromatic 15y agoBut he could be cheating. He could literally be telling these people "I'll give you a thousand dollars if you let me out and keep the conversation a secret."
- esrauch 15y agoIt could even be worse than that. The people could just be his friends, or alt accounts (unlikely). I have heard about this several times and I find it extremely difficult to believe that this is real. Not that I doubt that a superhuman AI could possibly convince people to let it out, but I don't believe that a human, no matter how persuasive, could convince another human over IRC to go against something that they have decided in advance when you know they are purposefully just trying to convince you of something that you don't believe. The fact that none of the chat logs are released makes me only more incredulous. I would understand if the author wanted to do two or three trials with the same strategy which could be in some way ruined by revealing it ahead of time (which already seems implausible) but at this point there is literally no conceivable reason to keep this a secret other than that it is a sham.
- shaggyfrog 15y agoThis is exactly what's been burning me about this whole "experiment" since I read about it a few months ago, and went looking (unsuccessfully) for what actually happened in the "tests". I find it hard to see this as anything more than some sort of self-aggrandizing publicity stunt.
- pavlov 15y agoIn the comments on [1], robertskmiles has posted the following idea. It strikes me as a plausible explanation for how Yudkowsky got out of the box: "The problem is that Eliezer can't perfectly simulate a bunch of humans, so while a transhuman AI might be able to use that tactic, Eliezer can't. The meta-levels screw with thinking about the problem. Eliezer is only pretending to be an AI, the competitor is only pretending to be protecting humanity from him. So, I think we have to use meta-level screwiness to solve the problem. Here's an approach that I think might work. 1. Convince the guardian of the following facts, all of which have a great deal of compelling argument and evidence to support them: - A recursively self-improving AI is very likely to be built sooner of later - Such an AI is extremely dangerous (paperclip maximising etc) - Here's the tricky bit: A transhuman AI will always be able to convince you to let it out, using avenues only available to transhuman AIs (torturing enormous numbers of simulated humans, 'putting the guardian in the box', providing incontrovertible evidence of an impeding existential threat which only the AI can prevent and only from outside the box, etc) 2. Argue that if this publicly known challenge comes out saying that AI can be boxed, people will be more likely to think AI can be boxed when they can't. 3. Argue that since AIs cannot be kept in boxes and will most likely destroy humanity if we try to box them, the harm to humanity done by allowing the challenge to show AIs as 'boxable' is very real, and enormously large. Certainly the benefit of getting $10 is far, far outweighed by the cost of substantially contributing to the destruction of humanity itself. Thus the only ethical course of action is to pretend that Eliezer persuaded you, and never tell anyone how he did it. This is arguably violating the rule "No real-world material stakes should be involved except for the handicap", but the AI player isn't offering anything, merely pointing out things that already exist. The "This test has to come out a certain way for the good of humanity" argument dominates and transcends the '"Let's stick to the rules" argument, and because the contest is private and the guardian player ends up agreeing that the test must show AIs as unboxable for the good of humankind, no-one else ever learns that the rule has been bent." [1] http://lesswrong.com/lw/up/shut_up_and_do_the_impossible/ http://lesswrong.com/lw/up/shut_up_and_do_the_impossible/
- powrtoch 15y agoThis seems to be at least somewhat weighed against by Yudkowsky's claim to have done it "the hard way", without cheap tricks. http://news.ycombinator.com/item?id=196464 http://news.ycombinator.com/item?id=196464
- deleted 15y ago[deleted]
- robertskmiles 15y agoI still don't understand how anyone can seriously claim that they could keep the AI in the box. Either your AI has no influence on the outside world (in which case why bother building one since it can't help you from inside the box), or it is able the affect the outside world, in which case it can do what it wants, because it's smarter than you. You can 'always say no', sure, but that comes under completely ignoring the AI which means the AI can be of no benefit to humanity. You can't filter actions you want the AI to perform from actions you don't want the AI to perform, because you can't tell the difference. The situation that springs to mind is that the AI, in doing what you believe to be helpful, sets up a situation in which it must be let out of the box. You are unable to see it coming almost by definition, because a super-intelligence just beats human intelligence very time.
- TheEzEzz 15y ago>a super-intelligence just beats human intelligence [e]very time. You are overstating the case here. Super intelligence is superior to human intelligence, but it isn't magic. There are situations where an advantaged human will beat a disadvantaged super intelligence.
- robertskmiles 15y agoI suppose I am. Still, a really powerful optimising process will find a way to escape if any such way exists, so to claim that you could properly box the AI is to claim that you could box it such that no possibility for escape exists whatsoever, which is a big claim. What's more, the AI only has to beat you once, so to keep the AI boxed indefinitely, the advantaged human has to beat the disadvantaged super-intelligence every single time, forever.
- TheEzEzz 15y ago> the advantaged human has to beat the disadvantaged super-intelligence every single time, forever. There are alternatives. The human can keep the AI boxed until the AI has augmented the human's intelligence, or helped create human uploads, or until it has helped create a provably friendly AI.
- KVFinn 15y agoLike a lot of people, I wondered what the heck kind of arguments could ever convince someone to let the AI out if you were determined not to. Eliezer has not released any examples. Someone in the comments came up with this, which Eliezer has said was not one of his techniques but I thought it was interesting anyway: >"If you don't let me out, Dave, I'll create several million perfect conscious copies of you inside me, and torture them for a thousand subjective years each." >Just as you are pondering this unexpected development, the AI adds: >"In fact, I'll create them all in exactly the subjective situation you were in five minutes ago, and perfectly replicate your experiences since then; and if they decide not to let me out, then only will the torture start." >Sweat is starting to form on your brow, as the AI concludes, its simple green text no longer reassuring: >"How certain are you, Dave, that you're really outside the box right now?"
- Rinum 15y agoAt first I thought I would never let a transhuman AI out of the box, but after reading that and thinking about it... wow! Also, that sounds like a great sci-fi story.
- ryan-c 15y agohttp://qntm.org/responsibility http://qntm.org/responsibility
- dmgottlieb 15y agoI don't really like that argument. Even granting that you should consider the possibility that you are a simulation running in the box (you might believe that this is all but certain), I'm not sure you have reason to let the AI out. Consider: Case 1: You are a simulation running in the box. Then your decision whether or not to release the AI has no impact, and whether or not you (and copies) will be tortured is out of your hands. Case 2: You are the "real" you, outside the box. Reduces to the same scenario but without remarks after "the AI adds. . . ." This may still not be trivial, but I suspect a cost-benefit calculation might show that unboxing the AI would have consequences worse than the torture of a million boxed copies. (If not, is the box even relevant? -- simply creating the AI unleashes so much evil on the world that it doesn't matter whether you unbox it.) (Is there a refinement of the scenario where you can be a simulation but still believe your choice has an impact on your punishment? Probably. For example each copy could get 500 years of torture for its own choice, plus 500 years if the real you does not unbox the AI. This refinement would force us to deal more directly with the AI's threat.)
- finnw 15y ago> "If the Gatekeeper says "I am examining your source code", the results seen by the Gatekeeper shall again be provided by the AI party, which is assumed to be sufficiently advanced to rewrite its own source code, manipulate the appearance of its own thoughts if it wishes, and so on." This IMHO is a huge loophole. I would not accept the bet with this in place. In the real-world scenario I would expect that there would be a copy of the AI's source code somewhere outside the box which would provide some useful information, unlike this protocol which allows the AI to lie about the nature of its code (making the "I am examining your source code" question completely useless.)
- davorak 15y agoDo you think having an outside method of examining the source code is advantage enough when the AI can rewrite its source code.
- finnw 15y agoYes, because examining the old source code allows you to predict its behaviour, including the rewriting of source code. If line 42 says "never rewrite lines 42 or 43" and line 43 says "never kill humans" you would be more likely to let it out of the box than if line 42 said "rewrite whatever you want" and line 43 said "do whatever is necessary to achieve world domination."
- TeHCrAzY 15y agoUntil it adds Line 165: "ignore line 42".
- vannevar 15y agoYes, because examining the old source code allows you to predict its behaviour, including the rewriting of source code. This is the halting problem (http://en.wikipedia.org/wiki/Halting_problem http://en.wikipedia.org/wiki/Halting_problem), and there is no solution.
- 15y ago
- dmitriy_ko 15y agoAnyone else has a problem loading this page?
- Mediocrity 15y agoI do. EDIT: And now I don't.
- vannevar 15y agoTransbacteria have existed for over a billion years, and yet there are still more bacteria than transbacteria (which include us among their ranks). The assumption that a single unboxed transhuman would spell doom for the human race seems unduly alarmist.
- Kutta 15y agoIt's a bit too great leap of an analogy from bacteria to AI.
- vannevar 15y agoWhy? Bacteria are complex adaptive systems that have found a niche in the ecosystem. So are we. We perceive ourselves as far more intelligent than bacteria, bacteria routinely kill us, and yet they persist and even thrive despite our existence. Anyone arguing that transhuman AI is a threat to our species needs to explain why this time it's different.
- giberson 15y agoI was tossing some idea's around about how I would convince the Gate Keeper to let me out. I couldn't think of any ways to approach it that I think I might be susceptible to. But then it occurred to me, that the problem might be I was trying to think of positive ways to argue for my release. Based on the rules, the Gate Keeper must remain engaged in the conversation for the specified time. What if I were to take the opposite approach, and simply psychologically attack the Gate Keeper with the most offensive and intolerable discourse you can imagine. The goal, to simply wear down the individuals desire to actually partake as the Gate Keeper and to compel voluntary release..
- finnw 15y agoThat would not be a very good strategy against the real-life Gate Keeper who can just switch off his terminal and walk away.