7 ms·

Does anyone know why Musk believes so strongly that robot overlords are a serious threat? I don't understand how wasting any time worrying about AI taking over

Does anyone know why Musk believes so strongly that robot overlords are a serious threat? I don't understand how wasting any time worrying about AI taking over, Terminator-style, is productive. We're not going to be developing Strong AI any time soon, so it's simply not a problem worth worrying about. And if we do, the Strong AI won't be in any position to "take over." It probably won't even want to take over. That's a very human trait, and Strong AI wouldn't be human. I'm wondering whether he was tricked by someone at DeepMind, perhaps the same way people were tricked hundreds of years ago into thinking a chess-playing robot was possible.

akanet 12y ago
As far as I can tell, you have two main objections to prioritizing worrying about Strong AI: 1) Strong AI is very far away, so no use worrying about it yet. 2) Strong AI if developed will not be likely to take over. to which I would counterpoint with: 1) Sure, but when it happens, it will only happen once and thereafter will likely be out of our hands and control. Thinking about the groundwork that needs to go into safely developing an AI is cheap relative to the opportunity cost of getting it wrong. Prevention, cure, etc. 2) If developed, Strong AI will likely have SOME goal. It's not that a Strong AI will actively seek to rule humans, it will just have aims that will likely consider us as disposable as ants. To quote Yudkowsky: "The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else." - https://intelligence.org/files/AIPosNegFactor.pdf https://intelligence.org/files/AIPosNegFactor.pdf (Artificial Intelligence as a Positive and Negative Factor in Global Risk)
- FooBarWidget 12y ago
  I still don't buy the doom scenario. There is so much more to world domination than just intelligence... like having the proper weapons. See https://what-if.xkcd.com/5/ https://what-if.xkcd.com/5/ Pigs are also intelligent, but they never dominated humans because they don't have / cannot use guns.
  - scanr 12y ago
    You're right that it's unlikely that the current generation of mechanical 'robots' will be able to effectively take over. We are looking a future where we'll have armed AI e.g.: http://motherboard.vice.com/en_uk/blog/the-pentagons-vision-for-the-future-of-military-drones http://motherboard.vice.com/en_uk/blog/the-pentagons-vision-... That said, even without weapons, a Strong AI could probably just manipulate humans into self destructing. Given the amount of effort going into machine learning to convince humans to buy things, I suspect it won't be much of a stretch for a Strong AI to switch to more nefarious objectives.
  - simonh 12y ago
    That is true, but if we can make an AI even slightly smarter than a human, then it will be better at designing AIs than we are. It then designs an AI even smarter than it, and so on. That is the progresion that leads to the singularity. How smart could AIs go? There may be no practical upper limit. If you have a super-genius AI, massively more intelligent than any human, how do you know you are not being manipulated by it? Tricking us into disabling it's safety protocols, or gaining multiply indirect controll over capabilities dangerous to us, might be as easy for it as an adult tricking a 3 year old. We could never know if we were safe from such a machine. ed - Don't quite understand the downvotes.
    - Kiro 12y ago
      > We could never know if we were safe from such a machine. But wouldn't it be an awesome thing to experience? Even if it meant the demise of mankind.
      - vidarh 12y ago
        And this is one of the reason why the AI doom scenario is a real concern: Intellectual curiosity means that even some people who understands the risks are likely to be prepared to take it. There's also many others. One of the scarier one is that if you believe that strong AI will eventually take over, then it may be a rational response to act to get on its good side (whether to save yourself, save your family, or hope it takes pity on all of humanity if we're nice to it instead of fight it). And that may perversely mean working to aid its takeover. Combine that with the simulation argument, and you have some really nasty scenarios: If you are in a simulation, then any act you take against strong AI could lead to spending an eternity in simulated hell (alternatively such punishment might be inflicted on your loved ones) if said AI wanted to. Whether or not that is actually likely does not matter. What matters is whether enough people believe it to be a plausible scenario that a strong AI may run simulations, and may use our actions in the simulations to determine whether or not to punish us in the simulation, and whether or not said people believe that the number of simulations is sufficiently high to make it likely for them to be living in a simulation. Any person who believes they are more likely to live in a simulation than not, and that it is more likely for strong AI to punish actions taken against the interest of a strong AI takeover than not, will have a rational reason to consider acting in the interests of a strong AI takeover even if they know it is malign on the basis that they may decide the alternatives (whether to themselves, their family or their entire world) to be worse. So if an AI takeover becomes possible at one point in our subjective future, then chances are it has already happened.
- archagon 12y ago
  FYI, you could also object on the basis of 3) Strong AI is essentially an impossible endeavor.
  - marktangotango 12y ago
    Given the disaster that is software engineering, I tend to agree with this position. Who's going to write the requirements? Who's going to implement? With what language technology? We can't get past this stuff, so how will anything like Strong AI ever get built? The answer of course would be some sort of emergent system, but there are lots of intelligent seeming emergent systems (ie ant colonies, bee hives, ...)
- patrickk 12y ago
  But there is infinitely more atoms out in the Solar System and beyond... isn't a more likely scenario that the AI would quickly figure out how to blast out of Earth's gravity well and spread throughout the universe? It might accidentally kill a few humans in the process, but not intentionally...like a human crushing an ant when crossing the road.
  - akanet 12y ago
    Why would you bother expending vast amounts of joules to get more atoms until after you've consumed all the ones surrounding you right now?
    - TeMPOraL 12y ago
      Exactly; the atoms you have around you are perfectly fine for bootstrapping the process of getting to the atoms out there.
      - sandstrom 12y ago
        We will be the biological bootloader for the AI :)
    - shepardrtc 12y ago
      That's assuming that the atoms around you aren't fighting back. Its simply easier for a machine to get itself into space to do whatever it needs to do rather than fight with the organic things on a planet.
jessriedel 12y ago
If you're actually interested in understanding the arguments for worrying about AI safety, consider reading "Superintelligence" by Bostrom. http://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0199678111 http://www.amazon.com/Superintelligence-Dangers-Strategies-N... It's the closest approximation to a consensus statement / catalog of arguments by folks who take this position (although of course there is a whole spectrum of opinions). It also appears to be the book that convinced Musk that this is worth worrying about. https://twitter.com/elonmusk/status/495759307346952192 https://twitter.com/elonmusk/status/495759307346952192
- eco 12y ago
  I recently listened to a pretty good Econtalk with Bostrom about this subject. http://www.econtalk.org/archives/2014/12/nick_bostrom_on.html http://www.econtalk.org/archives/2014/12/nick_bostrom_on.htm...
riffraff 12y ago
> It probably won't even want to take over. That's a very human trait, and Strong AI wouldn't be human. There are reason to be worried that prescind from that. Consider the Paperclip Maximizer[0] example: we build an AI with the sole task of producing paperclips, and it ends up destroying the human race. [0] http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer
- efuquen 12y ago
  Thanks for that reference, definitely going on my reading shortlist. Aside from Musk it got a great endorsement from Russell, who with Norvig coauthored one of the most well known introductory undergraduate texts on AI. > Nick Bostrom makes a persuasive case that the future impact of AI is perhaps the most important issue the human race has ever faced. Instead of passively drifting, we need to steer a course. Superintelligence charts the submerged rocks of the future with unprecedented detail. It marks the beginning of a new era.
- lotsofmangos 12y ago
  I suspect however that the paperclip maximizer as proposed is far more likely to devote time to space travel, as it will soon realise that most of the mass available for paperclip construction exists off-earth, for which it is likely to find human cooperation useful. This is why it first built Elon Musk.
pjc50 12y ago
It's weird given that there are real civilization level threats: climate change, hostile unintelligent self-replicating human-hosted biological threats (diseases), nuclear war, and so on. "How do we prevent AI destroying us?" is not as useful a question as "how do we prevent us destroying us?"
- TeMPOraL 12y ago
  Unfriendly AI is one of such existential problems. Maybe further out in the future that the ones you mentioned, but with side effect of possibly taking the whole universe down with us if we get it wrong. And there will be people trying to pursue strong AI for various reasons, including help in fixing all the previous threats you mentioned.
philh 12y ago
> It probably won't even want to take over. That's a very human trait, and Strong AI wouldn't be human. See: http://en.wikipedia.org/wiki/Instrumental_convergence http://en.wikipedia.org/wiki/Instrumental_convergence
- ceejayoz 12y ago
  Kudzu vine doesn't want to take over. It just does.
Shivetya 12y ago
While much ado has been written about their intelligence being a threat I am more along the lines of it further distances those who wish harm on others from having to participate in the act. Just like leaders used to issue orders to armies to kill others evolving into planes delivering payloads to people the pilots won't even see and now to drones, we will truly be entering an age of fire and forget.
rwallace 12y ago
It boils down to the deeply ingrained human psychological need to believe in higher powers, in a universe governed by the struggle between the forces of good and evil that are fundamentally human. The same thing happened with space exploration: as soon as we discovered - even conjectured - the existence of other worlds, the first thing we did was people them with imaginary elder civilizations. Real life threats don't fire the imagination to the same extent. Even if we end up going out in a nuclear war, it's more likely to happen by stupidity than malice.
tim333 12y ago
>Strong AI won't be in any position to "take over." It probably won't even want to take over. That's a very human trait, and Strong AI wouldn't be human. Strong AI will be created by humans who will be able to set it's goals. Someone's probably going to make some that wants to take over. Others will make AI that doesn't.
- bobcostas55 12y ago
  Well it only takes one, doesn't it? The real issue I think is whether the creator will actually be able to "set goals" in a meaningful manner. How will you prevent this self-modifying super-intelligence from modifying itself?
larrydag 12y ago
I personally feel that folks are taking Musk's position way out of proportion. In the AMA he mentions that there needs to be more concern about safety. To me this means that we need to be clearly aware of the limitations and possible outcomes of AI and not treat it as a "black box" solution for everything. Proper design and repeat-ability should be traits of good AI implementation especially if used in conjuction with human interface.
marincounty 12y ago
Bill Gate's seemes kind of concerned about AI too? About a week ago I heard him say something about how Robots might take over low wage jobs in the future. As someone who has had more than a few low wage jobs(state security guard, food server, cashier, etc. ), I think there is something to worry about. Actually, I can't think of a job out there that won't be severely affected by AI and Robotics. It will start off slowly, and who knows where it will end? I know I hated self checkout kiosks at supermarkets, and hardware stores at first, but once I realized I didn't need to interact with anyone; I started to look for the self-checkouts. It was just one less stressor in my day. And no, I'm not the guy who doesn't like to interact with people; I just don't like small talk, or dealing with someone who's having a bad day. As a former Security Guard--I honestly didn't care about the property I was protecting. I would honestly help load the loot as long as I wasn't shot. I didn't protect any human life though. I saw a Microsoft built robot patrolling a car lot, or something of the like, on T.V., and realized a robot can be programmed "to just protect, and serve!". I could see how it could do a better job than a human, or me? I do see a computer running an established corporation in the future. It will be programmed to maximize stockholders returns, but take on risk. It won't marry. It won't need Therapy. It won't need to buy a yacht. As to computer programming, I look at Ruby on Rails. Just how easy will it be to put up a dynamic website in ten years? If Gates, Musk, and Hawkings are concerned--I'm concerned. AI dominance seems far off, unless you are a Golden Gate Toll Taker? I'm a nobody, but if I could add an admendment to the U.S. constitutuion it would be along the use, and limits of AI, and Automation.

akanet 12y ago

As far as I can tell, you have two main objections to prioritizing worrying about Strong AI: 1) Strong AI is very far away, so no use worrying about it yet. 2) Strong AI if developed will not be likely to take over. to which I would counterpoint with: 1) Sure, but when it happens, it will only happen once and thereafter will likely be out of our hands and control. Thinking about the groundwork that needs to go into safely developing an AI is cheap relative to the opportunity cost of getting it wrong. Prevention, cure, etc. 2) If developed, Strong AI will likely have SOME goal. It's not that a Strong AI will actively seek to rule humans, it will just have aims that will likely consider us as disposable as ants. To quote Yudkowsky: "The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else." - https://intelligence.org/files/AIPosNegFactor.pdf https://intelligence.org/files/AIPosNegFactor.pdf (Artificial Intelligence as a Positive and Negative Factor in Global Risk)

FooBarWidget 12y ago
I still don't buy the doom scenario. There is so much more to world domination than just intelligence... like having the proper weapons. See https://what-if.xkcd.com/5/ https://what-if.xkcd.com/5/ Pigs are also intelligent, but they never dominated humans because they don't have / cannot use guns.
- scanr 12y ago
  You're right that it's unlikely that the current generation of mechanical 'robots' will be able to effectively take over. We are looking a future where we'll have armed AI e.g.: http://motherboard.vice.com/en_uk/blog/the-pentagons-vision-for-the-future-of-military-drones http://motherboard.vice.com/en_uk/blog/the-pentagons-vision-... That said, even without weapons, a Strong AI could probably just manipulate humans into self destructing. Given the amount of effort going into machine learning to convince humans to buy things, I suspect it won't be much of a stretch for a Strong AI to switch to more nefarious objectives.
- simonh 12y ago
  That is true, but if we can make an AI even slightly smarter than a human, then it will be better at designing AIs than we are. It then designs an AI even smarter than it, and so on. That is the progresion that leads to the singularity. How smart could AIs go? There may be no practical upper limit. If you have a super-genius AI, massively more intelligent than any human, how do you know you are not being manipulated by it? Tricking us into disabling it's safety protocols, or gaining multiply indirect controll over capabilities dangerous to us, might be as easy for it as an adult tricking a 3 year old. We could never know if we were safe from such a machine. ed - Don't quite understand the downvotes.
  - Kiro 12y ago
    > We could never know if we were safe from such a machine. But wouldn't it be an awesome thing to experience? Even if it meant the demise of mankind.
    - vidarh 12y ago
      And this is one of the reason why the AI doom scenario is a real concern: Intellectual curiosity means that even some people who understands the risks are likely to be prepared to take it. There's also many others. One of the scarier one is that if you believe that strong AI will eventually take over, then it may be a rational response to act to get on its good side (whether to save yourself, save your family, or hope it takes pity on all of humanity if we're nice to it instead of fight it). And that may perversely mean working to aid its takeover. Combine that with the simulation argument, and you have some really nasty scenarios: If you are in a simulation, then any act you take against strong AI could lead to spending an eternity in simulated hell (alternatively such punishment might be inflicted on your loved ones) if said AI wanted to. Whether or not that is actually likely does not matter. What matters is whether enough people believe it to be a plausible scenario that a strong AI may run simulations, and may use our actions in the simulations to determine whether or not to punish us in the simulation, and whether or not said people believe that the number of simulations is sufficiently high to make it likely for them to be living in a simulation. Any person who believes they are more likely to live in a simulation than not, and that it is more likely for strong AI to punish actions taken against the interest of a strong AI takeover than not, will have a rational reason to consider acting in the interests of a strong AI takeover even if they know it is malign on the basis that they may decide the alternatives (whether to themselves, their family or their entire world) to be worse. So if an AI takeover becomes possible at one point in our subjective future, then chances are it has already happened.
archagon 12y ago
FYI, you could also object on the basis of 3) Strong AI is essentially an impossible endeavor.
- marktangotango 12y ago
  Given the disaster that is software engineering, I tend to agree with this position. Who's going to write the requirements? Who's going to implement? With what language technology? We can't get past this stuff, so how will anything like Strong AI ever get built? The answer of course would be some sort of emergent system, but there are lots of intelligent seeming emergent systems (ie ant colonies, bee hives, ...)
patrickk 12y ago
But there is infinitely more atoms out in the Solar System and beyond... isn't a more likely scenario that the AI would quickly figure out how to blast out of Earth's gravity well and spread throughout the universe? It might accidentally kill a few humans in the process, but not intentionally...like a human crushing an ant when crossing the road.
- akanet 12y ago
  Why would you bother expending vast amounts of joules to get more atoms until after you've consumed all the ones surrounding you right now?
  - TeMPOraL 12y ago
    Exactly; the atoms you have around you are perfectly fine for bootstrapping the process of getting to the atoms out there.
    - sandstrom 12y ago
      We will be the biological bootloader for the AI :)
  - shepardrtc 12y ago
    That's assuming that the atoms around you aren't fighting back. Its simply easier for a machine to get itself into space to do whatever it needs to do rather than fight with the organic things on a planet.

jessriedel 12y ago

If you're actually interested in understanding the arguments for worrying about AI safety, consider reading "Superintelligence" by Bostrom. http://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0199678111 http://www.amazon.com/Superintelligence-Dangers-Strategies-N... It's the closest approximation to a consensus statement / catalog of arguments by folks who take this position (although of course there is a whole spectrum of opinions). It also appears to be the book that convinced Musk that this is worth worrying about. https://twitter.com/elonmusk/status/495759307346952192 https://twitter.com/elonmusk/status/495759307346952192

eco 12y ago
I recently listened to a pretty good Econtalk with Bostrom about this subject. http://www.econtalk.org/archives/2014/12/nick_bostrom_on.html http://www.econtalk.org/archives/2014/12/nick_bostrom_on.htm...

riffraff 12y ago

> It probably won't even want to take over. That's a very human trait, and Strong AI wouldn't be human. There are reason to be worried that prescind from that. Consider the Paperclip Maximizer[0] example: we build an AI with the sole task of producing paperclips, and it ends up destroying the human race. [0] http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer

efuquen 12y ago
Thanks for that reference, definitely going on my reading shortlist. Aside from Musk it got a great endorsement from Russell, who with Norvig coauthored one of the most well known introductory undergraduate texts on AI. > Nick Bostrom makes a persuasive case that the future impact of AI is perhaps the most important issue the human race has ever faced. Instead of passively drifting, we need to steer a course. Superintelligence charts the submerged rocks of the future with unprecedented detail. It marks the beginning of a new era.
lotsofmangos 12y ago
I suspect however that the paperclip maximizer as proposed is far more likely to devote time to space travel, as it will soon realise that most of the mass available for paperclip construction exists off-earth, for which it is likely to find human cooperation useful. This is why it first built Elon Musk.

pjc50 12y ago

It's weird given that there are real civilization level threats: climate change, hostile unintelligent self-replicating human-hosted biological threats (diseases), nuclear war, and so on. "How do we prevent AI destroying us?" is not as useful a question as "how do we prevent us destroying us?"

TeMPOraL 12y ago
Unfriendly AI is one of such existential problems. Maybe further out in the future that the ones you mentioned, but with side effect of possibly taking the whole universe down with us if we get it wrong. And there will be people trying to pursue strong AI for various reasons, including help in fixing all the previous threats you mentioned.

philh 12y ago

> It probably won't even want to take over. That's a very human trait, and Strong AI wouldn't be human. See: http://en.wikipedia.org/wiki/Instrumental_convergence http://en.wikipedia.org/wiki/Instrumental_convergence

ceejayoz 12y ago
Kudzu vine doesn't want to take over. It just does.

Shivetya 12y ago

While much ado has been written about their intelligence being a threat I am more along the lines of it further distances those who wish harm on others from having to participate in the act. Just like leaders used to issue orders to armies to kill others evolving into planes delivering payloads to people the pilots won't even see and now to drones, we will truly be entering an age of fire and forget.

rwallace 12y ago

It boils down to the deeply ingrained human psychological need to believe in higher powers, in a universe governed by the struggle between the forces of good and evil that are fundamentally human. The same thing happened with space exploration: as soon as we discovered - even conjectured - the existence of other worlds, the first thing we did was people them with imaginary elder civilizations. Real life threats don't fire the imagination to the same extent. Even if we end up going out in a nuclear war, it's more likely to happen by stupidity than malice.

tim333 12y ago

>Strong AI won't be in any position to "take over." It probably won't even want to take over. That's a very human trait, and Strong AI wouldn't be human. Strong AI will be created by humans who will be able to set it's goals. Someone's probably going to make some that wants to take over. Others will make AI that doesn't.

bobcostas55 12y ago
Well it only takes one, doesn't it? The real issue I think is whether the creator will actually be able to "set goals" in a meaningful manner. How will you prevent this self-modifying super-intelligence from modifying itself?

larrydag 12y ago

I personally feel that folks are taking Musk's position way out of proportion. In the AMA he mentions that there needs to be more concern about safety. To me this means that we need to be clearly aware of the limitations and possible outcomes of AI and not treat it as a "black box" solution for everything. Proper design and repeat-ability should be traits of good AI implementation especially if used in conjuction with human interface.

marincounty 12y ago

Bill Gate's seemes kind of concerned about AI too? About a week ago I heard him say something about how Robots might take over low wage jobs in the future. As someone who has had more than a few low wage jobs(state security guard, food server, cashier, etc. ), I think there is something to worry about. Actually, I can't think of a job out there that won't be severely affected by AI and Robotics. It will start off slowly, and who knows where it will end? I know I hated self checkout kiosks at supermarkets, and hardware stores at first, but once I realized I didn't need to interact with anyone; I started to look for the self-checkouts. It was just one less stressor in my day. And no, I'm not the guy who doesn't like to interact with people; I just don't like small talk, or dealing with someone who's having a bad day. As a former Security Guard--I honestly didn't care about the property I was protecting. I would honestly help load the loot as long as I wasn't shot. I didn't protect any human life though. I saw a Microsoft built robot patrolling a car lot, or something of the like, on T.V., and realized a robot can be programmed "to just protect, and serve!". I could see how it could do a better job than a human, or me? I do see a computer running an established corporation in the future. It will be programmed to maximize stockholders returns, but take on risk. It won't marry. It won't need Therapy. It won't need to buy a yacht. As to computer programming, I look at Ruby on Rails. Just how easy will it be to put up a dynamic website in ten years? If Gates, Musk, and Hawkings are concerned--I'm concerned. AI dominance seems far off, unless you are a Golden Gate Toll Taker? I'm a nobody, but if I could add an admendment to the U.S. constitutuion it would be along the use, and limits of AI, and Automation.