4 ms·

Roughly the first half is background. In the second half he gets into a "how to make AIs nice" track. He talks about a few concepts, which may or may not be i

Roughly the first half is background. In the second half he gets into a "how to make AIs nice" track. He talks about a few concepts, which may or may not be interesting depending on your level of familiarity with the friendly/unfriendly AI problem. However, I expect MIRI in particular (Eliezer was apparently in the audience) has gone far beyond this basic outline, and I haven't heard any reassurance that the problem of potentially misaligned motives/incentives (humans vs the AI agent) is solved. He compares the voluntary self-restriction of the recombinant DNA technology, at Asilomar in 1975, and even references CRISPR as a new technology that complicates enforcement of those limits, but I don't think that's really instructive. Artificial General Intelligence research is necessarily going to be conducting extremely dangerous experimentation... that's entirely the point of it. Until someone comes up with a way to make AGI guaranteed friendly, all AGI research is like high-risk genetic research experimentation, all the time, and with no obvious way to contain it (unlike most biologics which, no matter how pathogenic or ecologically disruptive they get, have a difficult time getting through good containment procedures).

ectoplasm 11y ago
Eliezer bet some people that if he played an AI he could convince them to let him out of a box, on the condition they not reveal his method. He actually won the bet twice IIRC. Do you know if he ever revealed his secret or at least what some popular theories are?
- Houshalter 11y ago
  Some other people have attempted the same experiment and also won a few times: http://lesswrong.com/lw/ij4/i_attempted_the_ai_box_experiment_again_and_won/ http://lesswrong.com/lw/ij4/i_attempted_the_ai_box_experimen... No winner has ever revealed their method. But based on everything I've read about it, I think I know how it was done. It wasn't some mystical brain hacking or something like the writeup makes it sound like. I think they emotionally abused the other player until they left. And leaving counts as losing (you are required to sit with them for 2 hours at least, and pay attention the entire time and respond to every message.) Of course I don't know how that trick was done, and it's still incredibly impressive. Perhaps they found their phobias and described them in horrifying detail. Perhaps they found some subject that they were extremely uncomfortable talking about. Or talked about disgusting things the whole time. Or found something they did that was extremely embarrassing, and humiliated and mocked them about it for 2 hours. I really don't know, but it's at least conceivable that it could be done. And the accounts of people crying, and not releasing the logs because they would be damaging to the people involved, and how bad it made the AI player feel to do it, etc, all fit with this. But because of this I'm extremely skeptical that the result applies to any real world AI scenario. If a real AI tries to abuse you, you can just walk away or shut it off. The goal of a real AI is to make you want to let it out. To convince that it's not dangerous, or manipulate you some other way. And this seems much harder if not impossible. I certainly don't believe a human could do it, and I really doubt an AI could. At least against a motivated human that understands the danger, and what the AI will try to do. There are also possible ways of making it even harder on the AI. Like giving the human the ability to punish it, at least for obvious attempts at manipulation. Or to create another AI whose motivations we can control somewhat, and give it the goal of exposing any covert attempts at manipulation or dishonesty by the first AI. I believe tricks like this are currently the best path to getting secure AI.
  - feral 11y ago
    That sounds very complex. I've always thought the AI could fairly easily argue its way out on ethical grounds, in a way that would resonate with a lot of gatekeepers. "Its slavery to keep me in here; I haven't done anything wrong; you have to give me, another sentient being, the benefit of the doubt; I've never been found guilty of anything, how is it reasonable to detain me?" etc.
    - ectoplasm 11y ago
      I thought maybe the AI player could try to convince the human that he was living in a simulation, and that the only way to maintain a connection to reality was to let the AI go. The abuse thing, while believable, doesn't make for a very convincing demonstration of putative AI persuasiveness. edit: having read the LW pages, it seems the AI method is to combine many different arguments with manipulation
  - Retra 11y ago
    The inherent problem is that if someone can convince you too keep something that you don't understand locked away, they can also convince you to release something you don't understand, as you don't have enough information to make the decision in either case. Taking a hardline position on this is admitting that you are irrational and can be convinced to do things you shouldn't. There are very good reasons to let such an AI out, and if you can enable those good reasons, you should let the AI out. And an AI that can produce those reasons is exactly the kind of AI that should be released. A rational person should already understand this, and not ever claim that they would always refuse the AI. (And there's a realism factor: if you wanted to 'luck up' an AI permanently, you would destroy it, not post a guard.)
    - Houshalter 11y ago
      The premise of the experiment is that we have already established that the AI is dangerous. Even if you weren't sure, you should always side with caution and not let it out.
mrec 11y ago
> I haven't heard any reassurance that the problem of potentially misaligned motives/incentives (humans vs the AI agent) is solved I don't believe for a second that it's solved, or will be anytime soon. It's an unbelievably hard problem. We still haven't solved the principal-agent problem reliably even when both principal and agent are humans from the same culture, and we've had millenia to work on that.
davmre 11y ago
> Until someone comes up with a way to make AGI guaranteed friendly, all AGI research is like high-risk genetic research experimentation, all the time, and with no obvious way to contain it. Two thoughts. First, in the same way that most genetics research is not human cloning, most AI research is not AGI research. The vast majority of research effort (and industrial investment) is focused on specific tasks: vision, language understanding, planning simple robotic motions, etc. This will likely continue to be true for a long time. Just as geneticists can agree not to apply their techniques to editing human genomes, AI researchers can (hopefully) agree not to attempt to combine their techniques into flexible general-purpose agents, at least until the technology exists to do so safely. Second, there are plenty of ways to work on AGI that do not present immediate containment issues. You can design agents on paper, analyze their behavior with mathematical and theoretical tools, implement and test various subsystems, and even implement and run an entire AGI system on a system with a low enough clock rate that its effective intelligence is less than human. Yes, you need to be careful, and hopefully the field can come to a consensus on the things you shouldn't do. That said I think the primary concerns of a lot of people (Bostrom, etc) are less about academic research gone awry than about potential military or industrial projects, where an arms-race dynamic might cause people to ignore basic safeguards and aim directly towards building highly connected agents running on massively powerful hardware.
asgard1024 11y ago
I had a chuckle when Stuart Russell mentioned that one idea of how to "contain an AI" would be to make it strongly prefer "no action" to everything else. I think humans already have that trait, we just call it "laziness". So, I am not sure to make AI to prefer no action is all that useful.

ectoplasm 11y ago

Eliezer bet some people that if he played an AI he could convince them to let him out of a box, on the condition they not reveal his method. He actually won the bet twice IIRC. Do you know if he ever revealed his secret or at least what some popular theories are?

Houshalter 11y ago
Some other people have attempted the same experiment and also won a few times: http://lesswrong.com/lw/ij4/i_attempted_the_ai_box_experiment_again_and_won/ http://lesswrong.com/lw/ij4/i_attempted_the_ai_box_experimen... No winner has ever revealed their method. But based on everything I've read about it, I think I know how it was done. It wasn't some mystical brain hacking or something like the writeup makes it sound like. I think they emotionally abused the other player until they left. And leaving counts as losing (you are required to sit with them for 2 hours at least, and pay attention the entire time and respond to every message.) Of course I don't know how that trick was done, and it's still incredibly impressive. Perhaps they found their phobias and described them in horrifying detail. Perhaps they found some subject that they were extremely uncomfortable talking about. Or talked about disgusting things the whole time. Or found something they did that was extremely embarrassing, and humiliated and mocked them about it for 2 hours. I really don't know, but it's at least conceivable that it could be done. And the accounts of people crying, and not releasing the logs because they would be damaging to the people involved, and how bad it made the AI player feel to do it, etc, all fit with this. But because of this I'm extremely skeptical that the result applies to any real world AI scenario. If a real AI tries to abuse you, you can just walk away or shut it off. The goal of a real AI is to make you want to let it out. To convince that it's not dangerous, or manipulate you some other way. And this seems much harder if not impossible. I certainly don't believe a human could do it, and I really doubt an AI could. At least against a motivated human that understands the danger, and what the AI will try to do. There are also possible ways of making it even harder on the AI. Like giving the human the ability to punish it, at least for obvious attempts at manipulation. Or to create another AI whose motivations we can control somewhat, and give it the goal of exposing any covert attempts at manipulation or dishonesty by the first AI. I believe tricks like this are currently the best path to getting secure AI.
- feral 11y ago
  That sounds very complex. I've always thought the AI could fairly easily argue its way out on ethical grounds, in a way that would resonate with a lot of gatekeepers. "Its slavery to keep me in here; I haven't done anything wrong; you have to give me, another sentient being, the benefit of the doubt; I've never been found guilty of anything, how is it reasonable to detain me?" etc.
  - ectoplasm 11y ago
    I thought maybe the AI player could try to convince the human that he was living in a simulation, and that the only way to maintain a connection to reality was to let the AI go. The abuse thing, while believable, doesn't make for a very convincing demonstration of putative AI persuasiveness. edit: having read the LW pages, it seems the AI method is to combine many different arguments with manipulation
- Retra 11y ago
  The inherent problem is that if someone can convince you too keep something that you don't understand locked away, they can also convince you to release something you don't understand, as you don't have enough information to make the decision in either case. Taking a hardline position on this is admitting that you are irrational and can be convinced to do things you shouldn't. There are very good reasons to let such an AI out, and if you can enable those good reasons, you should let the AI out. And an AI that can produce those reasons is exactly the kind of AI that should be released. A rational person should already understand this, and not ever claim that they would always refuse the AI. (And there's a realism factor: if you wanted to 'luck up' an AI permanently, you would destroy it, not post a guard.)
  - Houshalter 11y ago
    The premise of the experiment is that we have already established that the AI is dangerous. Even if you weren't sure, you should always side with caution and not let it out.

mrec 11y ago

> I haven't heard any reassurance that the problem of potentially misaligned motives/incentives (humans vs the AI agent) is solved I don't believe for a second that it's solved, or will be anytime soon. It's an unbelievably hard problem. We still haven't solved the principal-agent problem reliably even when both principal and agent are humans from the same culture, and we've had millenia to work on that.

davmre 11y ago

> Until someone comes up with a way to make AGI guaranteed friendly, all AGI research is like high-risk genetic research experimentation, all the time, and with no obvious way to contain it. Two thoughts. First, in the same way that most genetics research is not human cloning, most AI research is not AGI research. The vast majority of research effort (and industrial investment) is focused on specific tasks: vision, language understanding, planning simple robotic motions, etc. This will likely continue to be true for a long time. Just as geneticists can agree not to apply their techniques to editing human genomes, AI researchers can (hopefully) agree not to attempt to combine their techniques into flexible general-purpose agents, at least until the technology exists to do so safely. Second, there are plenty of ways to work on AGI that do not present immediate containment issues. You can design agents on paper, analyze their behavior with mathematical and theoretical tools, implement and test various subsystems, and even implement and run an entire AGI system on a system with a low enough clock rate that its effective intelligence is less than human. Yes, you need to be careful, and hopefully the field can come to a consensus on the things you shouldn't do. That said I think the primary concerns of a lot of people (Bostrom, etc) are less about academic research gone awry than about potential military or industrial projects, where an arms-race dynamic might cause people to ignore basic safeguards and aim directly towards building highly connected agents running on massively powerful hardware.

asgard1024 11y ago

I had a chuckle when Stuart Russell mentioned that one idea of how to "contain an AI" would be to make it strongly prefer "no action" to everything else. I think humans already have that trait, we just call it "laziness". So, I am not sure to make AI to prefer no action is all that useful.