4 ms·

For those interested in the "AI threat" discussion, here's an excellent thread on Quora about it with answers from Andrew Ng, Yoshua Bengio, and Pedro Domingos

For those interested in the "AI threat" discussion, here's an excellent thread on Quora about it with answers from Andrew Ng, Yoshua Bengio, and Pedro Domingos (along with other random opinions): https://www.quora.com/Is-AI-an-existential-threat-to-humanity https://www.quora.com/Is-AI-an-existential-threat-to-humanit... As a side note, Quora has done some really excellent AMAs with a bunch of machine learning researchers, including Ng, Bengio, researchers at Google, and many more.

forgotpwtomain 11y ago
> For those interested in the "AI threat" discussion, here's an excellent thread on Quora about it with answers from Andrew Ng, Yoshua Bengio, and Pedro Domingos (along with other random opinions): https://www.quora.com/Is-AI-an-existential-threat-to-humanit.. https://www.quora.com/Is-AI-an-existential-threat-to-humanit.... I read Andrew Ng's answer on quora - but I'm not sure it is really addressing the stronger arguments or concerns that would suggest 'AI may be existential threat to humanity' as much as a straw-man of the 'evil super-intelligence', with all respect, it's hard to call this an interesting or involved answer.
- solipsism 11y ago
  Why would he address those concerns? He was pretty clear that he believes it's silly to even think about the "problem", since it's so far from being a threat. In fact, I would guess that fear of the "AI threat" roughly correlates with ignorance of the state of AI. And it might correlate with number of science fiction movies watched.
  - forgotpwtomain 11y ago
    You think that the numerous experts who signed this letter expressing concerns: http://futureoflife.org/ai-open-letter/ http://futureoflife.org/ai-open-letter/ have these concerns from watching too many sci-fi movies about 'evil super-intelligence'?
    - solipsism 11y ago
      Have you ever actually read the letter? It's not a dooms-day letter. It doesn't talk about impending doom. It says basically, "there is potential for great power, let's wield it responsibly." This is not controversial. I would sign this letter and I don't think AI poses an existential threat.
      - forgotpwtomain 11y ago
        That was rather the point of my first post? That those whom have these concerns are mostly not worried about the fantastical version of 'evil super-intelligence' (although they are worried that AI could be an existential-threat) - of course you can address the most extreme-position presented and call it silly, but I don't think that's adding much to my understanding of the problem. For example I think a very serious concern is surrendering human decision making to machine-learning in the areas where big data-sets can be very widely applicable (crime, insurance, employment, immigration, finance) - loosing control of our own systems, to perhaps even provably better decision making is a totally not fantastic possibility. But if the response to this is a malicious super-intelligence won't send a robot with automatic weapons to your house in the next hundred years, I'm supposed to feel reassured?
        solipsism 11y ago
        You're missing what I'm saying. There's no reason to think that most people who signed that letter have any concerns. The letter is written such that signing it is basically an affirmation that we should use AI to help society. You started by wondering why Ng didn't address the concerns. It's because experts don't tend to be concerned. The letter was your evidence to the contrary, and it is not convincing.
        forgotpwtomain 11y ago
        > You started by wondering why Ng didn't address the concerns. It's because experts don't tend to be concerned. While not everyone who signed is necessarily concerned about 'existential dangers to humanity' or is an expert specifically in AI, for some signers both are definitely true - for an example you can cross reference: https://en.wikipedia.org/wiki/Existential_risk_from_artificial_general_intelligence https://en.wikipedia.org/wiki/Existential_risk_from_artifici...
- spangry 11y ago
  That link leads to a 404. That aside, it would be disappointing if it's addressing the 'evil killer robots' straw-man rather than the 'increasingly efficient optimiser that kills everyone as a by-product of its optimisation process' issue. Even if we give strong AI seemingly benign 'goals' (optimisation targets), there's a non-trivial chance it will result in the extinction of the human race. One example I've seen: imagine if we invented strong AI and gave it the goal 'eradicate cancer in humans'. The most efficient way to achieve this goal would probably be to vaporize all currently alive humans, thus eradicating cancer in humans for all time.
Houshalter 11y ago
Here's a survey of AI experts: http://www.nickbostrom.com/papers/survey.pdf http://www.nickbostrom.com/papers/survey.pdf >We thus designed a brief question- naire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that high- level machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity.
- argonaut 11y ago
  I see this survey cited a lot. I'd rather not get into a "my experts are better than your experts" argument (going on in the other thread here), but I really don't like Bostrom's survey. There are two really glaring issues: 1) Bostrom surveyed participants in conferences, not people presenting papers. Anyone can attend a conference if they buy a ticket, expert or not. 2) Bostrom decided to survey: the conferences "Philosophy and Theory of AI," "AGI," and "AGI-Impacts," all of them organized by himself or his colleague. He also emailed the mailing list of "Members of the Greek Association for Artificial Intelligence." Neither of these conferences are known in ML (or technical, for that matter). I have never heard of this association. Just about the only legitimate methodology was emailing the top 100 AI researchers according to Microsoft Academic search (only 29 responded). It would have much more weight to me if it were, say, a survey of people presenting papers at NIPS and ICML (the top 2 machine learning conferences). Maybe throw in ICLR, CVPR (top deep learning conference, top CV conference with ML focus). Because, like it or not, the only reason people fear super-intelligence is because of the advances being made in machine learning, and by extension, deep learning. Not because of advances (there are few) in classic AI.
- narrator 11y ago
  Asking about technology in 2075 is ridiculous. If you had asked people in 1916 about 1975, nobody would have gotten anything right.
  - Houshalter 11y ago
edanm 11y ago
It's a very interesting interesting thread. I will point out that there is no representative there of the other side of that discussion, namely that AI is a threat we should be devoting resources to solving.
- diskcat 11y ago
  Well a bunch of AI researchers are not gonna say AI is gonna kill us is 5 years. If they did, they would lose their jobs. If a self-improving AI could exist, it would be exponential in its self-improvement. So just like a supercritical nuclear reactor, it could conceivably go from undetectable level of activity to supreme intelligence within a very short span of time. It would be like the nuclear engineer at chernobyl saying "is only 100MW comrade, is of not problem"
  - GFK_of_xmaspast 11y ago
    > it would be exponential in its self-improvement Why do you think that is?
    - diskcat 11y ago
      because its ability to improve its intelligence is based on its intelligence i.e. exponential.
      - umanwizard 11y ago
        That assumes the marginal difficulty of further improvement stays constant as intelligence increases.
      - GFK_of_xmaspast 11y ago
        In addition to umanwizard's comment, you're assuming that there are no external constraints involved. There are plenty of non-exponential growth curves out there, like https://en.wikipedia.org/wiki/Logistic_function https://en.wikipedia.org/wiki/Logistic_function and https://en.wikipedia.org/wiki/Gompertz_function https://en.wikipedia.org/wiki/Gompertz_function , which you can imagine as being at least as probable as an exponential rate.

forgotpwtomain 11y ago

> For those interested in the "AI threat" discussion, here's an excellent thread on Quora about it with answers from Andrew Ng, Yoshua Bengio, and Pedro Domingos (along with other random opinions): https://www.quora.com/Is-AI-an-existential-threat-to-humanit.. https://www.quora.com/Is-AI-an-existential-threat-to-humanit.... I read Andrew Ng's answer on quora - but I'm not sure it is really addressing the stronger arguments or concerns that would suggest 'AI may be existential threat to humanity' as much as a straw-man of the 'evil super-intelligence', with all respect, it's hard to call this an interesting or involved answer.

solipsism 11y ago
Why would he address those concerns? He was pretty clear that he believes it's silly to even think about the "problem", since it's so far from being a threat. In fact, I would guess that fear of the "AI threat" roughly correlates with ignorance of the state of AI. And it might correlate with number of science fiction movies watched.
- forgotpwtomain 11y ago
  You think that the numerous experts who signed this letter expressing concerns: http://futureoflife.org/ai-open-letter/ http://futureoflife.org/ai-open-letter/ have these concerns from watching too many sci-fi movies about 'evil super-intelligence'?
  - solipsism 11y ago
    Have you ever actually read the letter? It's not a dooms-day letter. It doesn't talk about impending doom. It says basically, "there is potential for great power, let's wield it responsibly." This is not controversial. I would sign this letter and I don't think AI poses an existential threat.
    - forgotpwtomain 11y ago
      That was rather the point of my first post? That those whom have these concerns are mostly not worried about the fantastical version of 'evil super-intelligence' (although they are worried that AI could be an existential-threat) - of course you can address the most extreme-position presented and call it silly, but I don't think that's adding much to my understanding of the problem. For example I think a very serious concern is surrendering human decision making to machine-learning in the areas where big data-sets can be very widely applicable (crime, insurance, employment, immigration, finance) - loosing control of our own systems, to perhaps even provably better decision making is a totally not fantastic possibility. But if the response to this is a malicious super-intelligence won't send a robot with automatic weapons to your house in the next hundred years, I'm supposed to feel reassured?
      - solipsism 11y ago
        You're missing what I'm saying. There's no reason to think that most people who signed that letter have any concerns. The letter is written such that signing it is basically an affirmation that we should use AI to help society. You started by wondering why Ng didn't address the concerns. It's because experts don't tend to be concerned. The letter was your evidence to the contrary, and it is not convincing.
        forgotpwtomain 11y ago
        > You started by wondering why Ng didn't address the concerns. It's because experts don't tend to be concerned. While not everyone who signed is necessarily concerned about 'existential dangers to humanity' or is an expert specifically in AI, for some signers both are definitely true - for an example you can cross reference: https://en.wikipedia.org/wiki/Existential_risk_from_artificial_general_intelligence https://en.wikipedia.org/wiki/Existential_risk_from_artifici...
spangry 11y ago
That link leads to a 404. That aside, it would be disappointing if it's addressing the 'evil killer robots' straw-man rather than the 'increasingly efficient optimiser that kills everyone as a by-product of its optimisation process' issue. Even if we give strong AI seemingly benign 'goals' (optimisation targets), there's a non-trivial chance it will result in the extinction of the human race. One example I've seen: imagine if we invented strong AI and gave it the goal 'eradicate cancer in humans'. The most efficient way to achieve this goal would probably be to vaporize all currently alive humans, thus eradicating cancer in humans for all time.

Houshalter 11y ago

Here's a survey of AI experts: http://www.nickbostrom.com/papers/survey.pdf http://www.nickbostrom.com/papers/survey.pdf >We thus designed a brief question- naire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that high- level machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity.

argonaut 11y ago
I see this survey cited a lot. I'd rather not get into a "my experts are better than your experts" argument (going on in the other thread here), but I really don't like Bostrom's survey. There are two really glaring issues: 1) Bostrom surveyed participants in conferences, not people presenting papers. Anyone can attend a conference if they buy a ticket, expert or not. 2) Bostrom decided to survey: the conferences "Philosophy and Theory of AI," "AGI," and "AGI-Impacts," all of them organized by himself or his colleague. He also emailed the mailing list of "Members of the Greek Association for Artificial Intelligence." Neither of these conferences are known in ML (or technical, for that matter). I have never heard of this association. Just about the only legitimate methodology was emailing the top 100 AI researchers according to Microsoft Academic search (only 29 responded). It would have much more weight to me if it were, say, a survey of people presenting papers at NIPS and ICML (the top 2 machine learning conferences). Maybe throw in ICLR, CVPR (top deep learning conference, top CV conference with ML focus). Because, like it or not, the only reason people fear super-intelligence is because of the advances being made in machine learning, and by extension, deep learning. Not because of advances (there are few) in classic AI.
narrator 11y ago
Asking about technology in 2075 is ridiculous. If you had asked people in 1916 about 1975, nobody would have gotten anything right.
- Houshalter 11y ago

edanm 11y ago

It's a very interesting interesting thread. I will point out that there is no representative there of the other side of that discussion, namely that AI is a threat we should be devoting resources to solving.

diskcat 11y ago
Well a bunch of AI researchers are not gonna say AI is gonna kill us is 5 years. If they did, they would lose their jobs. If a self-improving AI could exist, it would be exponential in its self-improvement. So just like a supercritical nuclear reactor, it could conceivably go from undetectable level of activity to supreme intelligence within a very short span of time. It would be like the nuclear engineer at chernobyl saying "is only 100MW comrade, is of not problem"
- GFK_of_xmaspast 11y ago
  > it would be exponential in its self-improvement Why do you think that is?
  - diskcat 11y ago
    because its ability to improve its intelligence is based on its intelligence i.e. exponential.
    - umanwizard 11y ago
      That assumes the marginal difficulty of further improvement stays constant as intelligence increases.
    - GFK_of_xmaspast 11y ago
      In addition to umanwizard's comment, you're assuming that there are no external constraints involved. There are plenty of non-exponential growth curves out there, like https://en.wikipedia.org/wiki/Logistic_function https://en.wikipedia.org/wiki/Logistic_function and https://en.wikipedia.org/wiki/Gompertz_function https://en.wikipedia.org/wiki/Gompertz_function , which you can imagine as being at least as probable as an exponential rate.