7 ms·

The Long-Term Future of Artificial Intelligence [video]

89 points by jcr 11y ago

mark_l_watson 11y ago
I listened to 10 minutes and bookmarked it. I had not heard any of his talks before - nice sense of humor! I liked his comment on how is is disingenuous for AI researchers to say that they don't believe real AI is possible (I am roughy paraphrasing him).
- mark_l_watson 11y ago
  EDIT: I watched the whole video last night - he has an interesting take on where AI research needs to go. Also, his article in the latest Communications of the ACM is very good, and I recommend it.
harshreality 11y ago
Roughly the first half is background. In the second half he gets into a "how to make AIs nice" track. He talks about a few concepts, which may or may not be interesting depending on your level of familiarity with the friendly/unfriendly AI problem. However, I expect MIRI in particular (Eliezer was apparently in the audience) has gone far beyond this basic outline, and I haven't heard any reassurance that the problem of potentially misaligned motives/incentives (humans vs the AI agent) is solved. He compares the voluntary self-restriction of the recombinant DNA technology, at Asilomar in 1975, and even references CRISPR as a new technology that complicates enforcement of those limits, but I don't think that's really instructive. Artificial General Intelligence research is necessarily going to be conducting extremely dangerous experimentation... that's entirely the point of it. Until someone comes up with a way to make AGI guaranteed friendly, all AGI research is like high-risk genetic research experimentation, all the time, and with no obvious way to contain it (unlike most biologics which, no matter how pathogenic or ecologically disruptive they get, have a difficult time getting through good containment procedures).
- ectoplasm 11y ago
  Eliezer bet some people that if he played an AI he could convince them to let him out of a box, on the condition they not reveal his method. He actually won the bet twice IIRC. Do you know if he ever revealed his secret or at least what some popular theories are?
  - Houshalter 11y ago
    Some other people have attempted the same experiment and also won a few times: http://lesswrong.com/lw/ij4/i_attempted_the_ai_box_experiment_again_and_won/ http://lesswrong.com/lw/ij4/i_attempted_the_ai_box_experimen... No winner has ever revealed their method. But based on everything I've read about it, I think I know how it was done. It wasn't some mystical brain hacking or something like the writeup makes it sound like. I think they emotionally abused the other player until they left. And leaving counts as losing (you are required to sit with them for 2 hours at least, and pay attention the entire time and respond to every message.) Of course I don't know how that trick was done, and it's still incredibly impressive. Perhaps they found their phobias and described them in horrifying detail. Perhaps they found some subject that they were extremely uncomfortable talking about. Or talked about disgusting things the whole time. Or found something they did that was extremely embarrassing, and humiliated and mocked them about it for 2 hours. I really don't know, but it's at least conceivable that it could be done. And the accounts of people crying, and not releasing the logs because they would be damaging to the people involved, and how bad it made the AI player feel to do it, etc, all fit with this. But because of this I'm extremely skeptical that the result applies to any real world AI scenario. If a real AI tries to abuse you, you can just walk away or shut it off. The goal of a real AI is to make you want to let it out. To convince that it's not dangerous, or manipulate you some other way. And this seems much harder if not impossible. I certainly don't believe a human could do it, and I really doubt an AI could. At least against a motivated human that understands the danger, and what the AI will try to do. There are also possible ways of making it even harder on the AI. Like giving the human the ability to punish it, at least for obvious attempts at manipulation. Or to create another AI whose motivations we can control somewhat, and give it the goal of exposing any covert attempts at manipulation or dishonesty by the first AI. I believe tricks like this are currently the best path to getting secure AI.
mooneater 11y ago
So much of these discussions are overly focussed on an general purpose AI singularity, and what the AI itself would do. That makes for interesting discussions, but I think a much more immediate concern, is what strong AIs will enable their owners to do. For example, nation states owning large fleets of robots, corporations owning powerful AIs attached to markets and industry. The first group or few groups of people to control a rather powerful AI would have incredible power and leverage over other humans. The danger here is not the AIs behaving "badly" of their own accord, but rather their owners instructing them to further their own selfish goals (which appears not to be uncommon behaviour among those in power) and succeeding wildly with complete political, military, and/or commercial domination. In fact, succeeding at any of these may well lead directly to succeeding at all of them.
- hyperion2010 11y ago
  I always wonder why people worry about AI when it is so abundantly clear that the real danger is other people and we are just projecting.
  - imaginenore 11y ago
    Because we're familiar with humans. Most humans don't want to hurt/kill other humans, nor do they have the power to do that on a large scale. We can't say that about an AI. If it decides to exterminate us, there's nothing much we can do.
    - espadrine 11y ago
      > If it decides to exterminate us, there's nothing much we can do. That's absurd. No inorganic intelligence ever created rivals ants in their ability to survive on Earth. They are way more energy-efficient, and way smarter in the survivalist sense, than any of them. If ants decide to exterminate us, there's nothing much we can do. Except they wouldn't succeed. But you probably think that we, humans, are much smarter than ants (which is arguable). Yet again, if we decided to exterminate ants, there's nothing much they can do, but we wouldn't succeed. They're too smart for us, and we're too smart for them, creating a situation where annihilation of either is impractical. I'm not sure why people consider inorganic intelligence differently from the way they consider organic intelligence. I wrote a bit about it here: http://espadrine.tumblr.com/post/119471459626/on-inorganic-intelligence http://espadrine.tumblr.com/post/119471459626/on-inorganic-i....
tmerr 11y ago
Russel makes a mistake at 30:00 when he says computers are "totally unable to play checkers". Excerpt from J Schaeffer et al (2007). Checkers is Solved: >In this paper we announce that checkers has been weakly solved. From the starting position (Fig. 1, top), we have a computational proof that checkers is a draw. The proof consists of an explicit strategy that never loses — the program can achieve at least a draw against any opponent, playing either the black or white pieces. Not the focus of the talk, just thought I'd point it out.
- binarymax 11y ago
  What he meant is that an AI built for playing chess cannot play checkers. The software for solving checkers is also equally useless in the context of chess. The distinction here is that there is no general purpose AI in existence. All the surprising leaps have been customized for specific tasks.
  - tmerr 11y ago
    Listening again in context you're 100% right! Disregard my previous comment.
humanfromearth 11y ago
It could be potentially a very stupid idea, but creating multiple AIs at the exact same time with similar resources available to them for development in different locations of the world would create a balance of power. What I'm trying to argue is that it kind of works for us even in the most aggressive situations: If you bomb me I bomb you. If one AI is super-aggressive then others could decide to stop it. If it wants to convert all matter for making paper clips other AIs would say: I don't want to be a paper clip and actually have the power to do something about it. Where as us.. we'd be powerless. In the case where all band against us, well.. we'd have many problems. All of this to say that developing one single super-human AI in a box is worse in my opinion.
- eloff 11y ago
  The Avogadro trilogy of books explore this idea
dmfdmf 11y ago
This video is pretty interesting in laying out the questions concerning AI and its implications. But not much new for anyone who has done any study of the questions but still worthwhile to hear it again in different forms and perspectives. One thing I notice about AI talks such as this one is that there is a lot of vague concepts and implicit assumptions (especially regarding values)and psychological projection going on when evaluating the implications of what AI will or will not do. To separate out our fears of failure or the fear of others more intelligent than ourselves someone needs to do the following thought experiment; What if we invented a pill that would make the next generation 10 times smarter than the current generation. If the average IQ is 100 then the next generation (of those who's parents opted to give them the pill) would have IQs of 1000 or an order of magnitude increase. Such a scenario would have all the "singularity" effects in a few generations while separating out all the confusion and problems of it being artificial intelligence and computers doing the thinking. Would you want this pill to be generally available or ban it? Would you give it to your kids? This approach would really force the AI discussion where it belongs. What are values and where do they come from? Is intelligence an unconditional value? Can values be defined rationally and scientifically and be proved? A major benefit to this aspect of AI research is that it will force the scientific study of values and ultimately to the rejection of the implicit assumption that values are subjective and not open to reason.

89 points by jcr 11y ago

I listened to 10 minutes and bookmarked it. I had not heard any of his talks before - nice sense of humor! I liked his comment on how is is disingenuous for AI researchers to say that they don't believe real AI is possible (I am roughy paraphrasing him).

mark_l_watson 11y ago
EDIT: I watched the whole video last night - he has an interesting take on where AI research needs to go. Also, his article in the latest Communications of the ACM is very good, and I recommend it.

harshreality 11y ago

Roughly the first half is background. In the second half he gets into a "how to make AIs nice" track. He talks about a few concepts, which may or may not be interesting depending on your level of familiarity with the friendly/unfriendly AI problem. However, I expect MIRI in particular (Eliezer was apparently in the audience) has gone far beyond this basic outline, and I haven't heard any reassurance that the problem of potentially misaligned motives/incentives (humans vs the AI agent) is solved. He compares the voluntary self-restriction of the recombinant DNA technology, at Asilomar in 1975, and even references CRISPR as a new technology that complicates enforcement of those limits, but I don't think that's really instructive. Artificial General Intelligence research is necessarily going to be conducting extremely dangerous experimentation... that's entirely the point of it. Until someone comes up with a way to make AGI guaranteed friendly, all AGI research is like high-risk genetic research experimentation, all the time, and with no obvious way to contain it (unlike most biologics which, no matter how pathogenic or ecologically disruptive they get, have a difficult time getting through good containment procedures).

ectoplasm 11y ago
Eliezer bet some people that if he played an AI he could convince them to let him out of a box, on the condition they not reveal his method. He actually won the bet twice IIRC. Do you know if he ever revealed his secret or at least what some popular theories are?
- Houshalter 11y ago
  Some other people have attempted the same experiment and also won a few times: http://lesswrong.com/lw/ij4/i_attempted_the_ai_box_experiment_again_and_won/ http://lesswrong.com/lw/ij4/i_attempted_the_ai_box_experimen... No winner has ever revealed their method. But based on everything I've read about it, I think I know how it was done. It wasn't some mystical brain hacking or something like the writeup makes it sound like. I think they emotionally abused the other player until they left. And leaving counts as losing (you are required to sit with them for 2 hours at least, and pay attention the entire time and respond to every message.) Of course I don't know how that trick was done, and it's still incredibly impressive. Perhaps they found their phobias and described them in horrifying detail. Perhaps they found some subject that they were extremely uncomfortable talking about. Or talked about disgusting things the whole time. Or found something they did that was extremely embarrassing, and humiliated and mocked them about it for 2 hours. I really don't know, but it's at least conceivable that it could be done. And the accounts of people crying, and not releasing the logs because they would be damaging to the people involved, and how bad it made the AI player feel to do it, etc, all fit with this. But because of this I'm extremely skeptical that the result applies to any real world AI scenario. If a real AI tries to abuse you, you can just walk away or shut it off. The goal of a real AI is to make you want to let it out. To convince that it's not dangerous, or manipulate you some other way. And this seems much harder if not impossible. I certainly don't believe a human could do it, and I really doubt an AI could. At least against a motivated human that understands the danger, and what the AI will try to do. There are also possible ways of making it even harder on the AI. Like giving the human the ability to punish it, at least for obvious attempts at manipulation. Or to create another AI whose motivations we can control somewhat, and give it the goal of exposing any covert attempts at manipulation or dishonesty by the first AI. I believe tricks like this are currently the best path to getting secure AI.

mooneater 11y ago

So much of these discussions are overly focussed on an general purpose AI singularity, and what the AI itself would do. That makes for interesting discussions, but I think a much more immediate concern, is what strong AIs will enable their owners to do. For example, nation states owning large fleets of robots, corporations owning powerful AIs attached to markets and industry. The first group or few groups of people to control a rather powerful AI would have incredible power and leverage over other humans. The danger here is not the AIs behaving "badly" of their own accord, but rather their owners instructing them to further their own selfish goals (which appears not to be uncommon behaviour among those in power) and succeeding wildly with complete political, military, and/or commercial domination. In fact, succeeding at any of these may well lead directly to succeeding at all of them.

hyperion2010 11y ago
I always wonder why people worry about AI when it is so abundantly clear that the real danger is other people and we are just projecting.
- imaginenore 11y ago
  Because we're familiar with humans. Most humans don't want to hurt/kill other humans, nor do they have the power to do that on a large scale. We can't say that about an AI. If it decides to exterminate us, there's nothing much we can do.
  - espadrine 11y ago
    > If it decides to exterminate us, there's nothing much we can do. That's absurd. No inorganic intelligence ever created rivals ants in their ability to survive on Earth. They are way more energy-efficient, and way smarter in the survivalist sense, than any of them. If ants decide to exterminate us, there's nothing much we can do. Except they wouldn't succeed. But you probably think that we, humans, are much smarter than ants (which is arguable). Yet again, if we decided to exterminate ants, there's nothing much they can do, but we wouldn't succeed. They're too smart for us, and we're too smart for them, creating a situation where annihilation of either is impractical. I'm not sure why people consider inorganic intelligence differently from the way they consider organic intelligence. I wrote a bit about it here: http://espadrine.tumblr.com/post/119471459626/on-inorganic-intelligence http://espadrine.tumblr.com/post/119471459626/on-inorganic-i....

tmerr 11y ago

Russel makes a mistake at 30:00 when he says computers are "totally unable to play checkers". Excerpt from J Schaeffer et al (2007). Checkers is Solved: >In this paper we announce that checkers has been weakly solved. From the starting position (Fig. 1, top), we have a computational proof that checkers is a draw. The proof consists of an explicit strategy that never loses — the program can achieve at least a draw against any opponent, playing either the black or white pieces. Not the focus of the talk, just thought I'd point it out.

binarymax 11y ago
What he meant is that an AI built for playing chess cannot play checkers. The software for solving checkers is also equally useless in the context of chess. The distinction here is that there is no general purpose AI in existence. All the surprising leaps have been customized for specific tasks.
- tmerr 11y ago
  Listening again in context you're 100% right! Disregard my previous comment.

humanfromearth 11y ago

It could be potentially a very stupid idea, but creating multiple AIs at the exact same time with similar resources available to them for development in different locations of the world would create a balance of power. What I'm trying to argue is that it kind of works for us even in the most aggressive situations: If you bomb me I bomb you. If one AI is super-aggressive then others could decide to stop it. If it wants to convert all matter for making paper clips other AIs would say: I don't want to be a paper clip and actually have the power to do something about it. Where as us.. we'd be powerless. In the case where all band against us, well.. we'd have many problems. All of this to say that developing one single super-human AI in a box is worse in my opinion.

eloff 11y ago
The Avogadro trilogy of books explore this idea

dmfdmf 11y ago

This video is pretty interesting in laying out the questions concerning AI and its implications. But not much new for anyone who has done any study of the questions but still worthwhile to hear it again in different forms and perspectives. One thing I notice about AI talks such as this one is that there is a lot of vague concepts and implicit assumptions (especially regarding values)and psychological projection going on when evaluating the implications of what AI will or will not do. To separate out our fears of failure or the fear of others more intelligent than ourselves someone needs to do the following thought experiment; What if we invented a pill that would make the next generation 10 times smarter than the current generation. If the average IQ is 100 then the next generation (of those who's parents opted to give them the pill) would have IQs of 1000 or an order of magnitude increase. Such a scenario would have all the "singularity" effects in a few generations while separating out all the confusion and problems of it being artificial intelligence and computers doing the thinking. Would you want this pill to be generally available or ban it? Would you give it to your kids? This approach would really force the AI discussion where it belongs. What are values and where do they come from? Is intelligence an unconditional value? Can values be defined rationally and scientifically and be proved? A major benefit to this aspect of AI research is that it will force the scientific study of values and ultimately to the rejection of the implicit assumption that values are subjective and not open to reason.