5 ms·

> On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to

> On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter This doesn't seem true. E.g. the signatories on the Future of Life Institute's recent open letter about AI progress and safety include: * Stuart Russell & Peter Norvig, co-authors of the #1 AI textbook * Tom Dietterich, AAAI President * Eric Horvitz, past AAAI President * Bart Selman, co-chair of AAAI presidential panel on long-term AI futures * Francesca Rossi, IJCAI President * All founders of Deep Mind and Vicarious, two leading AI companies * Yann LeCun, head of Facebook AI * Geoffrey Hinton, top academic ML researcher * Yoshua Bengio, top academic ML researcher ...and many more.

msutherl 12y ago
That letter is not about "superhuman machine intelligence", it is about "ensuring that AI remains robust and beneficial." Among the threats cited are: autonomous vehicles, machines that need to make ethical decisions, autonomous weapons, surveillance, and issues of verification, validity, and control. Words like "singularity", "superhuman", and "AGI" do not appear.
- jkramar 12y ago
  Superintelligence and intelligence explosion are mentioned as valuable things to investigate.
  - msutherl 12y ago
    Ah, turns out you're right: Finally, research on the possibility of superintelligent machines or rapid, sustained self-improvement (“intelligence explosion”) has been highlighted by past and current projects on the future of AI as potentially valuable to the project of maintaining reliable control in the long term. The the AAAI 2008–09 Presidential Panel on Long-Term AI Futures’ “Subgroup on Pace, Concerns, and Control” stated that: "There was overall skepticism about the prospect of an intelligence explosion... Nevertheless, there was a shared sense that additional research would be valuable on methods for understanding and verifying the range of behaviors of complex computational systems to minimize unexpected outcomes. Some panelists recommended that more research needs to be done to better define “intelligence explosion,” and also to better formulate different classes of such accelerating intelligences. Technical work would likely lead to enhanced understanding of the likelihood of such phenomena, and the nature, risks, and overall outcomes associated with different conceived variants." Stanford’s One-Hundred Year Study of Artificial Intelligence includes “Loss of Control of AI systems” as an area of study, specifically highlighting concerns over the possibility that: "we could one day lose control of AI systems via the rise of superintelligences that do not act in accordance with human wishes – and that such powerful systems would threaten humanity. Are such dystopic outcomes possible? If so, how might these situations arise? ...What kind of investments in research should be made to better understand and to address the possibility of the rise of a dangerous superintelligence or the occurrence of an 'intelligence explosion'?" Research in this area could include any of the long-term research priorities listed above, as well as theoretical and forecasting work on intelligence explosion and superintelligence, and could extend or critique existing approaches begun by groups such as the Machine Intelligence Research Institute.
pavedwalden 12y ago
The tone of that letter was "Here's a general framework for responsibly building more powerful AI systems", whereas the lay press is reporting "Musk predicts Skynet in 5 years" http://futureoflife.org/static/data/documents/research_priorities.pdf http://futureoflife.org/static/data/documents/research_prior...
lazaroclapp 12y ago
To be fair "AI safety" is very much an issue even without human-level or post-human AI. I mean, there are far more "mundane" problems such as what happens when a self-driving car gets in a collision? or what do we do after factory workers, service workers and truck drivers are replaced by (dumber-than-human) machines for those jobs?
AndrewKemendo 12y ago
Sorry Luke. As usual your fearmongering is not going to get far. For example - MIRI skeptic and famed AGI researcher Ben Goertzel signed the letter (as did I) but not because we are primarily concerned with the threat. Rather: I signed the document because I wanted to signal to Max Tegmark and his colleagues that I am in favor of research aimed at figuring out how to maximize the odds that AGI systems are robust and beneficial. The letter helps give visibility and to that end better avenues for funding for AGI projects. [1]http://multiverseaccordingtoben.blogspot.hk/ http://multiverseaccordingtoben.blogspot.hk/
- jeffreyrogers 12y ago
  I'm glad to see that people like you are replying to some of the misinformation that occurs in threads like these. It's a shame that on HN the people with the least knowledge are the most likely to cast their opinions in broad sweeping statements grounded in hyperbole rather than fact, particularly when the topics of AI and ML come up. In fact a number of the signatories that Luke cites are quoted as saying on no uncertain terms that fear of AI is widely out of proportion to its capabilities now and for the foreseeable future. The most prominent I'm aware of saying thing to this effect are Yoshua Bengio, Geoff Hinton, and Yann LeCun.
  - AndrewKemendo 12y ago
    At the risk of falling into "ad hominem" territory, it is disheartening that Altman (Or Muehlhauser or most of the MIRI people for that matter) gets any play at all on this topic. The fact that Altman thanks Luke in his blog for review just gives you a sense of the "blind leading the blind" here. I think however that says more about the lack of awareness on the part of the readers than anything. That combination is a sure sign of a very narrow field of study with big political footprint and is typical of similar fields such as climatology, macroeconomics etc... where rank amateur thinks they have insight to the same degree as expert by virtue of "just thinking about it."
    - rl3 12y ago
      >At the risk of falling into "ad hominem" territory, ... I wouldn't worry, you and the parent already have solid footing in "appeal to authority" territory.

msutherl 12y ago

That letter is not about "superhuman machine intelligence", it is about "ensuring that AI remains robust and beneficial." Among the threats cited are: autonomous vehicles, machines that need to make ethical decisions, autonomous weapons, surveillance, and issues of verification, validity, and control. Words like "singularity", "superhuman", and "AGI" do not appear.

jkramar 12y ago
Superintelligence and intelligence explosion are mentioned as valuable things to investigate.
- msutherl 12y ago
  Ah, turns out you're right: Finally, research on the possibility of superintelligent machines or rapid, sustained self-improvement (“intelligence explosion”) has been highlighted by past and current projects on the future of AI as potentially valuable to the project of maintaining reliable control in the long term. The the AAAI 2008–09 Presidential Panel on Long-Term AI Futures’ “Subgroup on Pace, Concerns, and Control” stated that: "There was overall skepticism about the prospect of an intelligence explosion... Nevertheless, there was a shared sense that additional research would be valuable on methods for understanding and verifying the range of behaviors of complex computational systems to minimize unexpected outcomes. Some panelists recommended that more research needs to be done to better define “intelligence explosion,” and also to better formulate different classes of such accelerating intelligences. Technical work would likely lead to enhanced understanding of the likelihood of such phenomena, and the nature, risks, and overall outcomes associated with different conceived variants." Stanford’s One-Hundred Year Study of Artificial Intelligence includes “Loss of Control of AI systems” as an area of study, specifically highlighting concerns over the possibility that: "we could one day lose control of AI systems via the rise of superintelligences that do not act in accordance with human wishes – and that such powerful systems would threaten humanity. Are such dystopic outcomes possible? If so, how might these situations arise? ...What kind of investments in research should be made to better understand and to address the possibility of the rise of a dangerous superintelligence or the occurrence of an 'intelligence explosion'?" Research in this area could include any of the long-term research priorities listed above, as well as theoretical and forecasting work on intelligence explosion and superintelligence, and could extend or critique existing approaches begun by groups such as the Machine Intelligence Research Institute.

pavedwalden 12y ago

The tone of that letter was "Here's a general framework for responsibly building more powerful AI systems", whereas the lay press is reporting "Musk predicts Skynet in 5 years" http://futureoflife.org/static/data/documents/research_priorities.pdf http://futureoflife.org/static/data/documents/research_prior...

lazaroclapp 12y ago

To be fair "AI safety" is very much an issue even without human-level or post-human AI. I mean, there are far more "mundane" problems such as what happens when a self-driving car gets in a collision? or what do we do after factory workers, service workers and truck drivers are replaced by (dumber-than-human) machines for those jobs?

AndrewKemendo 12y ago

Sorry Luke. As usual your fearmongering is not going to get far. For example - MIRI skeptic and famed AGI researcher Ben Goertzel signed the letter (as did I) but not because we are primarily concerned with the threat. Rather: I signed the document because I wanted to signal to Max Tegmark and his colleagues that I am in favor of research aimed at figuring out how to maximize the odds that AGI systems are robust and beneficial. The letter helps give visibility and to that end better avenues for funding for AGI projects. [1]http://multiverseaccordingtoben.blogspot.hk/ http://multiverseaccordingtoben.blogspot.hk/

jeffreyrogers 12y ago
I'm glad to see that people like you are replying to some of the misinformation that occurs in threads like these. It's a shame that on HN the people with the least knowledge are the most likely to cast their opinions in broad sweeping statements grounded in hyperbole rather than fact, particularly when the topics of AI and ML come up. In fact a number of the signatories that Luke cites are quoted as saying on no uncertain terms that fear of AI is widely out of proportion to its capabilities now and for the foreseeable future. The most prominent I'm aware of saying thing to this effect are Yoshua Bengio, Geoff Hinton, and Yann LeCun.
- AndrewKemendo 12y ago
  At the risk of falling into "ad hominem" territory, it is disheartening that Altman (Or Muehlhauser or most of the MIRI people for that matter) gets any play at all on this topic. The fact that Altman thanks Luke in his blog for review just gives you a sense of the "blind leading the blind" here. I think however that says more about the lack of awareness on the part of the readers than anything. That combination is a sure sign of a very narrow field of study with big political footprint and is typical of similar fields such as climatology, macroeconomics etc... where rank amateur thinks they have insight to the same degree as expert by virtue of "just thinking about it."
  - rl3 12y ago
    >At the risk of falling into "ad hominem" territory, ... I wouldn't worry, you and the parent already have solid footing in "appeal to authority" territory.