4 ms·

They're talking past each other because "AI" has become a useless term. Musk's comments are about a potential (maybe conscious) artificial super intelligence a

They're talking past each other because "AI" has become a useless term. Musk's comments are about a potential (maybe conscious) artificial super intelligence and the risks that could come from that if we enable one somehow without understanding the preconditions. Zuckerberg is just talking about "AI" in the sense that machine learning can solve a valuable class of problems. The super intelligence risk could be serious or not - it's hard to know since we haven't really dealt with an intelligence that wasn't created through natural selection with those biases for survival. We also don't really understand how consciousness works either. One risk of AI is that if Musk's argument is right we won't be as 'lucky' as we were with nuclear weapons. Enriched uranium is hard to get so nuclear weapons are easier to control and hard for an individual to make. Turning on an AI or copying the code for one and spreading it would probably be easier. If it does end up being dangerous that's not a great situation.

arethuza 9y ago
"Turning on an AI or copying the code for one and spreading it would probably be easiear. If it does end up being dangerous that's bad." That scenario reminds me of the opening of Vernor Vinge's A Fire Upon the Deep.
6d6b73 9y ago
>Turning on an AI or copying the code for one and spreading it would probably be easiear. What people never mention in any discussion with AI is that the AI will be as limited by physical world as we are. AI It's not just code, it's also the hardware that will run the code. That hardware has theoretical and practical limits, and requires a lot of power. It's not like AI running on some peta/exascale supercomputer will be able to jump to your laptop and take it over and spread itself to every electronic device in the world.
- mindcrime 9y ago
  Exactly. And that's one big reason I find so many of these "doomsday" scenarios less than useful. They're largely based on a view of AI that's rooted in science-fiction, not reality.
- AndrewKemendo 9y ago
  It's not like AI running on some peta/exascale supercomputer will be able to jump to your laptop and take it over and spread itself to every electronic device in the world That's exactly a proposed scenario - with the form of "jumping" being any kind of communication. See: The AI Box problem. https://en.wikipedia.org/wiki/AI_box https://en.wikipedia.org/wiki/AI_box
  - 6d6b73 9y ago
    If the AI requires a supercomputer to "exists" it can only jump to another supercomputer. It will not be able to spread to your Arduino.
    - AndrewKemendo 9y ago
      Or it could figure out how to distribute it's computing and then distribute itself a piece at a time off of the "supercomputer."
      - 6d6b73 9y ago
        Or maybe it could decide to move to another universe and take it over.. There are physical limitations to what AI even in the form of super-intelligence can do. While theoretically it could distribute itself to other computers, it first would need to learn a lot everything from math to computer security, which itself would take implies that it would have unlimited resources and time to do all of that.. Even if that superAI did learn all of the skills necessary to take over many computers (it would need to learn about architectures, operating systems, firewalls, antivirus, compilers etc) it would still be limited by the latency of the network so not all operations would make sense in a distributed system. And of course it would need to be so smart that nobody would notice that there is increase of "weird" traffic on the internet, that suddenly power usage would increase as computers would do all these calculations instead of sitting idle waiting for user to open fb.com. To take over the world such AI would need to learn everything we already know about the world, not only including math and physics, but also including sociology and human behavior. While AI would be busy taking over the world, you would have to assume that we would be sitting, doing nothing, and that we would not be smart enough to figure out that something is going on. And when we did we would do nothing to stop it. Is it possible, sure.. Likely? hell no.
- fossuser 9y ago
  I think the main argument against this is once you're in the world of "super intelligence" you're severely outmatched. If for some reason it had an interest in staying on it might not be easy to stop.
  - 6d6b73 9y ago
    super intelligence will not happen overnight. AI will keep evolving with us and before it's truly conscious it will give us better understanding of math, physics, medicine.. This will help the AI grow, but at the same time we will be growing with it. It will be a symbiotic relationship.
    - fossuser 9y ago
      Not necessarily, there's the idea of an "intelligence explosion" - basically that while it may take a while to figure out the initial conditions it might be able to self improve rapidly from that point. Also consciousness may not be required (or possibly even preferable).
      - 6d6b73 9y ago
        "intelligence explosion" is also limited by the physical world. AI won't be able to come up with a better understanding of physics simply by reading academic journals. Yes it might have some good ideas, and find some of the stuff we're missing by simply connecting the dots, but that in no way will cause "intelligence explosion". Just think how much money and time we have to spend to test just a few of all of the theories in physics. And no, AI will not be able to suddenly come up with better theories simply by simulating physical world.
      - TheGrassyKnoll 9y ago
        "Colossus: The Forbin Project" (1970) http://www.imdb.com/title/tt0064177/?ref_=fn_al_tt_1 http://www.imdb.com/title/tt0064177/?ref_=fn_al_tt_1
euyyn 9y ago
But how can one think that Mark's understanding of "sci-fi AI" is limited? We're so far from any of it that at this point it's all speculation about what could or could not happen. If I have a theory that humans will evolve into separate species, I can't say someone else's understanding on the matter is limited.
- fossuser 9y ago
  Limited in a sense that he's saying Musk is being irresponsible and then talking about the simpler machine learning problems that can be solved [1] when Musk's warning is really about the super intelligence piece. The main problem is the term AI is overloaded and overused so it creates confusion. Musk is not against the machine learning approach to solving a new class of useful problems (though replacing a lot of current human jobs and the political risk associated with that is an issue to think about) [1] http://www.cnbc.com/2017/07/24/mark-zuckerberg-elon-musks-doomsday-ai-predictions-are-irresponsible.html http://www.cnbc.com/2017/07/24/mark-zuckerberg-elon-musks-do...
  - euyyn 9y ago
    "Has limited understanding" is definitely not the wording I'd chose to mean "is talking of something different," but I see how it could be.
- JCzynski 9y ago
  Surveys of the field consistently show a majority expecting less than a century, and people focused on the problem less than 50.
naveen99 9y ago
But AI is not just bits in cold storage. It would need a lot of computing power and or bandwidth. it would need money and power also.
WhatIsDukkha 9y ago
It's not useless as a term but AI has become a catchall for anxiety about the future. I'll add another set of concerns you see in these discussions. People know that the current version of predatory Capitalism we've created so far would happily mass unemploy everyone (possibly all the white collar workers reading this much sooner then you think). This would leave no workers and no consumers with money outside of the ultrawealthy. I agree. I don't trust the current version of capitalism to program these AI of the future. I think they will burn the house down in their greed.
- gmarx 9y ago
  I'll add another concern which is already occurring (the thing I'm concerned about, not the concern, although I guess the concern is already occurring) AI is being used as a workaround to existing anti-discrimination laws. The core of anti-discrimination law is that it doesn't matter if you think on the average my group of people is a worse credit risk or more likely to rob your taxi or more likely to trash your rental property; you need to judge me by the risk of the general population. AI allows you to create a black box which will inevitably find proxies for race and spit out a higher risk for some races than others. Race won't directly appear in the model but what does will be strongly correlated with race.
convexfunction 9y ago
> We also don't really understand how consciousness works either. Sort of a nitpick, but people concerned about AI risk usually don't postulate that an AI needs to be """conscious""" to be dangerous; it just needs to be a thing that can apply a lot of optimization power in unexpected ways toward goals that aren't perfectly aligned with what humans generally want.
- circlefavshape 9y ago
  Oh! Good insight! ... but we already have that, right? Undesirable outcomes from rulesets that govern the behaviour of a large number of (conscious or unconscious) actors are a pretty standard feature of civilisation
  - convexfunction 9y ago
    Yes, we do have that already, and it's kind of odd that people who see no point in thinking hard about risks to humans from this particular type of inhuman optimization process would readily agree that other types of inhuman optimization processes (with much better understood risk profiles and limitations) can be very harmful. :)
    - mercer 9y ago
      Perhaps the 'singularity' type stuff makes a lot of people disengage? So far I'm inclined to think that the whole 'sentient AI mucking things up' is jumping the gun (to put it mildly). I might very well be wrong though. However, it's only because of HN that I even bother to make the distinction between that particular fear (realistic or otherwise) and the much more directly useful discussion of how AI-type stuff (even just facebook feed algorithms) can be dangerous or harmful.
    - circlefavshape 9y ago
      I don't think it's odd. I agree that optimising for the wrong thing (or, probably, for any particular thing) can be harmful, especially when it's done on a very large scale. AI-as-we-know-it doesn't introduce a new class of risk, it just makes some of the optimisations faster. I don't see any point in thinking about AGI risk until we get some experimental evidence that AGI is possible
ankushnarula 9y ago
It doesn't even need to be a superintelligence - Musk explains a scenario that isn't very far on the horizon here at the Governor's meeting: https://youtu.be/OYJ89vE-QfQ?t=2959 https://youtu.be/OYJ89vE-QfQ?t=2959 (from 51:45 to 53:50)
sebular 9y ago
I think that they're actually talking right at each other, but Zuckerberg is thinking of the possibilities, while Musk is thinking of the implementations. "AI is the future" really means that Zuckerberg sees the massive potential of automation, and how it's going to change the face of the world. "We should be worried about AI" really means that Musk has had to think long and hard about the fact that the software written by Tesla is navigating living human beings around obstacles at high speeds. This isn't about some accidental super intelligence that becomes self-aware and murders all humans, it's about the consequences of applying today's standards of "let's ship this code" to technology that we're expected to entrust with our lives. This is two things, combined: "AI" means that code will have more complex emergent behavior that's harder to debug than current code, which is already nearly impossible to completely debug. "AI" also means that this code will run very complex systems that often have powerful hardware components, unfortunately including weapons. To put it another way: Both men agree that the code of the near future is going to make a massive leap in effectiveness and real-world applications. But Musk has to grapple with being held morally accountable for that code, while the only thing Zuckerberg's ever had to care about is performance. Say "AI" to Mark Zuckerberg and he sees all of the money in smarter data mining and fun home appliances. Say "AI" to Elon Musk and he has nightmares of deploying a wireless update that causes Teslas to confuse people wearing green clothing with green traffic lights.

arethuza 9y ago

"Turning on an AI or copying the code for one and spreading it would probably be easiear. If it does end up being dangerous that's bad." That scenario reminds me of the opening of Vernor Vinge's A Fire Upon the Deep.

6d6b73 9y ago

>Turning on an AI or copying the code for one and spreading it would probably be easiear. What people never mention in any discussion with AI is that the AI will be as limited by physical world as we are. AI It's not just code, it's also the hardware that will run the code. That hardware has theoretical and practical limits, and requires a lot of power. It's not like AI running on some peta/exascale supercomputer will be able to jump to your laptop and take it over and spread itself to every electronic device in the world.

mindcrime 9y ago
Exactly. And that's one big reason I find so many of these "doomsday" scenarios less than useful. They're largely based on a view of AI that's rooted in science-fiction, not reality.
AndrewKemendo 9y ago
It's not like AI running on some peta/exascale supercomputer will be able to jump to your laptop and take it over and spread itself to every electronic device in the world That's exactly a proposed scenario - with the form of "jumping" being any kind of communication. See: The AI Box problem. https://en.wikipedia.org/wiki/AI_box https://en.wikipedia.org/wiki/AI_box
- 6d6b73 9y ago
  If the AI requires a supercomputer to "exists" it can only jump to another supercomputer. It will not be able to spread to your Arduino.
  - AndrewKemendo 9y ago
    Or it could figure out how to distribute it's computing and then distribute itself a piece at a time off of the "supercomputer."
    - 6d6b73 9y ago
      Or maybe it could decide to move to another universe and take it over.. There are physical limitations to what AI even in the form of super-intelligence can do. While theoretically it could distribute itself to other computers, it first would need to learn a lot everything from math to computer security, which itself would take implies that it would have unlimited resources and time to do all of that.. Even if that superAI did learn all of the skills necessary to take over many computers (it would need to learn about architectures, operating systems, firewalls, antivirus, compilers etc) it would still be limited by the latency of the network so not all operations would make sense in a distributed system. And of course it would need to be so smart that nobody would notice that there is increase of "weird" traffic on the internet, that suddenly power usage would increase as computers would do all these calculations instead of sitting idle waiting for user to open fb.com. To take over the world such AI would need to learn everything we already know about the world, not only including math and physics, but also including sociology and human behavior. While AI would be busy taking over the world, you would have to assume that we would be sitting, doing nothing, and that we would not be smart enough to figure out that something is going on. And when we did we would do nothing to stop it. Is it possible, sure.. Likely? hell no.
fossuser 9y ago
I think the main argument against this is once you're in the world of "super intelligence" you're severely outmatched. If for some reason it had an interest in staying on it might not be easy to stop.
- 6d6b73 9y ago
  super intelligence will not happen overnight. AI will keep evolving with us and before it's truly conscious it will give us better understanding of math, physics, medicine.. This will help the AI grow, but at the same time we will be growing with it. It will be a symbiotic relationship.
  - fossuser 9y ago
    Not necessarily, there's the idea of an "intelligence explosion" - basically that while it may take a while to figure out the initial conditions it might be able to self improve rapidly from that point. Also consciousness may not be required (or possibly even preferable).
    - 6d6b73 9y ago
      "intelligence explosion" is also limited by the physical world. AI won't be able to come up with a better understanding of physics simply by reading academic journals. Yes it might have some good ideas, and find some of the stuff we're missing by simply connecting the dots, but that in no way will cause "intelligence explosion". Just think how much money and time we have to spend to test just a few of all of the theories in physics. And no, AI will not be able to suddenly come up with better theories simply by simulating physical world.
    - TheGrassyKnoll 9y ago
      "Colossus: The Forbin Project" (1970) http://www.imdb.com/title/tt0064177/?ref_=fn_al_tt_1 http://www.imdb.com/title/tt0064177/?ref_=fn_al_tt_1

euyyn 9y ago

But how can one think that Mark's understanding of "sci-fi AI" is limited? We're so far from any of it that at this point it's all speculation about what could or could not happen. If I have a theory that humans will evolve into separate species, I can't say someone else's understanding on the matter is limited.

fossuser 9y ago
Limited in a sense that he's saying Musk is being irresponsible and then talking about the simpler machine learning problems that can be solved [1] when Musk's warning is really about the super intelligence piece. The main problem is the term AI is overloaded and overused so it creates confusion. Musk is not against the machine learning approach to solving a new class of useful problems (though replacing a lot of current human jobs and the political risk associated with that is an issue to think about) [1] http://www.cnbc.com/2017/07/24/mark-zuckerberg-elon-musks-doomsday-ai-predictions-are-irresponsible.html http://www.cnbc.com/2017/07/24/mark-zuckerberg-elon-musks-do...
- euyyn 9y ago
  "Has limited understanding" is definitely not the wording I'd chose to mean "is talking of something different," but I see how it could be.
JCzynski 9y ago
Surveys of the field consistently show a majority expecting less than a century, and people focused on the problem less than 50.

naveen99 9y ago

But AI is not just bits in cold storage. It would need a lot of computing power and or bandwidth. it would need money and power also.

WhatIsDukkha 9y ago

It's not useless as a term but AI has become a catchall for anxiety about the future. I'll add another set of concerns you see in these discussions. People know that the current version of predatory Capitalism we've created so far would happily mass unemploy everyone (possibly all the white collar workers reading this much sooner then you think). This would leave no workers and no consumers with money outside of the ultrawealthy. I agree. I don't trust the current version of capitalism to program these AI of the future. I think they will burn the house down in their greed.

gmarx 9y ago
I'll add another concern which is already occurring (the thing I'm concerned about, not the concern, although I guess the concern is already occurring) AI is being used as a workaround to existing anti-discrimination laws. The core of anti-discrimination law is that it doesn't matter if you think on the average my group of people is a worse credit risk or more likely to rob your taxi or more likely to trash your rental property; you need to judge me by the risk of the general population. AI allows you to create a black box which will inevitably find proxies for race and spit out a higher risk for some races than others. Race won't directly appear in the model but what does will be strongly correlated with race.

convexfunction 9y ago

> We also don't really understand how consciousness works either. Sort of a nitpick, but people concerned about AI risk usually don't postulate that an AI needs to be """conscious""" to be dangerous; it just needs to be a thing that can apply a lot of optimization power in unexpected ways toward goals that aren't perfectly aligned with what humans generally want.

circlefavshape 9y ago
Oh! Good insight! ... but we already have that, right? Undesirable outcomes from rulesets that govern the behaviour of a large number of (conscious or unconscious) actors are a pretty standard feature of civilisation
- convexfunction 9y ago
  Yes, we do have that already, and it's kind of odd that people who see no point in thinking hard about risks to humans from this particular type of inhuman optimization process would readily agree that other types of inhuman optimization processes (with much better understood risk profiles and limitations) can be very harmful. :)
  - mercer 9y ago
    Perhaps the 'singularity' type stuff makes a lot of people disengage? So far I'm inclined to think that the whole 'sentient AI mucking things up' is jumping the gun (to put it mildly). I might very well be wrong though. However, it's only because of HN that I even bother to make the distinction between that particular fear (realistic or otherwise) and the much more directly useful discussion of how AI-type stuff (even just facebook feed algorithms) can be dangerous or harmful.
  - circlefavshape 9y ago
    I don't think it's odd. I agree that optimising for the wrong thing (or, probably, for any particular thing) can be harmful, especially when it's done on a very large scale. AI-as-we-know-it doesn't introduce a new class of risk, it just makes some of the optimisations faster. I don't see any point in thinking about AGI risk until we get some experimental evidence that AGI is possible

ankushnarula 9y ago

It doesn't even need to be a superintelligence - Musk explains a scenario that isn't very far on the horizon here at the Governor's meeting: https://youtu.be/OYJ89vE-QfQ?t=2959 https://youtu.be/OYJ89vE-QfQ?t=2959 (from 51:45 to 53:50)

sebular 9y ago

I think that they're actually talking right at each other, but Zuckerberg is thinking of the possibilities, while Musk is thinking of the implementations. "AI is the future" really means that Zuckerberg sees the massive potential of automation, and how it's going to change the face of the world. "We should be worried about AI" really means that Musk has had to think long and hard about the fact that the software written by Tesla is navigating living human beings around obstacles at high speeds. This isn't about some accidental super intelligence that becomes self-aware and murders all humans, it's about the consequences of applying today's standards of "let's ship this code" to technology that we're expected to entrust with our lives. This is two things, combined: "AI" means that code will have more complex emergent behavior that's harder to debug than current code, which is already nearly impossible to completely debug. "AI" also means that this code will run very complex systems that often have powerful hardware components, unfortunately including weapons. To put it another way: Both men agree that the code of the near future is going to make a massive leap in effectiveness and real-world applications. But Musk has to grapple with being held morally accountable for that code, while the only thing Zuckerberg's ever had to care about is performance. Say "AI" to Mark Zuckerberg and he sees all of the money in smarter data mining and fun home appliances. Say "AI" to Elon Musk and he has nightmares of deploying a wireless update that causes Teslas to confuse people wearing green clothing with green traffic lights.