6 ms·

There's this: http://www.nickbostrom.com/papers/survey.pdf http://www.nickbostrom.com/papers/survey.pdf >We thus designed a brief questionnaire and distributed

There's this: http://www.nickbostrom.com/papers/survey.pdf http://www.nickbostrom.com/papers/survey.pdf >We thus designed a brief questionnaire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that high-level machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity. This survey is a few years old. Discussion and knowledge about AI risk has increased considerably since then. And AI itself has made remarkable progress in that time as well. It's amazing how much progress there has been in AI in just the last 5 years. Who knows where it will be in 40 years. Look where computer technology was 40 years ago.

j2kun 10y ago
Is there an established, rigorous definition of "high level machine intelligence"? Or even a concrete list of sufficient criteria? Every single discussion I've encountered leaves the notion undefined, or just "I'll know it when I see it"
- Houshalter 10y ago
  Artificial General Intelligence is roughly defined as a machine that can do all the things a human can do. For example, an AI capable of doing AI research and programming computers, would be AGI. The Turing test (as Turing originally described it, not garbage like chatbot competitions) is the most widely accepted standard of AGI. Superintelligence goes well beyond that. A machine with cognitive abilities far beyond humans. I don't think such a machine is very unlikely even in the near future. It's unreasonable to believe that humans are the pinnacle of intelligence. We are just the very first intelligent creature to evolve. Our brains are heavily resource constrained by size and energy. And neurons are many orders of magnitude slower than transistors. And also far larger and less compact.
  - rtx 10y ago
    So how will this intelligent thing come about.
    - Houshalter 10y ago
      Once we have AIs as smart as humans, they can do AI research as good or better than human researchers. And they can make AIs that are even better, which in turn can make even better AIs, and so on. Dumb evolution was able to create human-level intelligence with just random mutations and natural selection. Surely human engineers can do better. But in the worst case, we could reverse engineer the human brain.
      - evgen 10y ago
        > Once we have AIs as smart as humans Whether a true 'generalist' AI is possible in the forseeable future is debatable. > they can do AI research as good or better than human researchers. Now your AI is not just a 'generalist' but rather a specialist in AI. A very big leap of faith has occurred here. This also presumes that the AIs are even capable of effective invention and improvisation rather than mimicry and optimization (the only two features we have seen from the very best of cutting-edge ML work so far.) > And they can make AIs that are even better, which in turn can make even better AIs, and so on. All of which is predicated on the limiting factor being software and not hardware. If it is the latter then these postulated AIs are hitting the same brick wall as humans and thinking about the problem faster or harder does not magically make the necessary hardware appear.
        Houshalter 10y ago
        >Whether a true 'generalist' AI is possible in the forseeable future is debatable. Sure, but see the survey I posted above. The rate of progress of AI is incredible. We will almost certainly be approaching human level in a few decades at most. >Now your AI is not just a 'generalist' but rather a specialist in AI. That's what general intelligence is. The ability to learn different specializations. AI researchers are not literally born as AI researchers and capable of nothing else. > This also presumes that the AIs are even capable of effective invention and improvisation rather than mimicry and optimization Why wouldn't they be? If they are generally intelligent and can do all the same tasks humans can do. Whats magical about invention that would prevent computers from ever doing it? >All of which is predicated on the limiting factor being software and not hardware. All the same arguments apply to hardware. Hardware has been improving exponentially for a much longer time than AI has. And I think hardware may already be close enough. Transistors are orders of magnitude faster and denser than biological synapses.
- AbrahamParangi 10y ago
  How about: "It'll know you when it sees you"
- quadcore 10y ago
  Maybe that's not the right question. What I understand from the article is that the threat is about being manipulated by computer algorithms. We don't really care the computers are actually intelligent and thinking or whatnot. What's bad is that we are manipulated by computers and they automates more and more areas of our lives each year.
- make3 10y ago
  futurism as a whole is 100% un-scientific
  - Houshalter 10y ago
    And does being unscientific make a thing wrong? Saying "we won't have strong AI in our lifetimes" is an equally strong claim that is also unscientific. You can't say "we can't know" because what does that even mean? That the outcome is 50/50? That's still pretty decent odds for something so significant!
    - make3 10y ago
      As a professional scientist in AI and deep learning, I see futurists making wild predictions about AI that I wouldn't see anyone serious in the field coming remotely close to making. I don't know how it is with other futurism fields like genetics, space travel and energy generation, but I expect that it's a similar situation. All I'm trying to say is, let's please keep a cool head and a wholly skeptical approach about all this.
      - Houshalter 10y ago
        But the survey I posted that started this discussion was a survey of experts in AI. Note that no one is claiming that skynet will appear any day now. Just that we will very likely have human-level AI in a few decades. I don't think that's a terribly wild or speculative claim.
    - make3 10y ago
      also please note I didn't say "we won't have strong AI in our lifetimes". Let's work on the actual developments specific of science instead of what we dream it to be. We want people to think about deep learning as a sophisticated pattern analysis algorithm instead of thinking that Skynet is going to start a coup any day now.
vectorpush 10y ago
Consider that 41% of those surveyed conclude that machines will never be able to "simulate learning and every other aspect of human intelligence". The next 41% conclude it is possible within an unbounded "more than 50 years". It seems to me that the vast majority of experts are pretty skeptical.
- Houshalter 10y ago
  That's not 41%, that's 4.1%. When asked when there would be a 90% of AI, the median answer was 2075.
  - vectorpush 10y ago
    No. It's definitely 41%. Look at page 6 of your link The earliest that machines will be able to simulate learning and every other aspect of human intelligence: Within 10 years 6 5% Between 11 and 25 years 3 2% Between 26 and 50 years 14 11% More than 50 years 50 41% Never 50 41% Totals 123 100%
    - Houshalter 10y ago
      That's in the prior work section. It's a much older survey of one specific conference.
      - gydfi 10y ago
        And was the other survey a comprehensive survey of everybody? The divergence in poll results seems to suggest we should put even less credence on polls here than usual.
        Houshalter 10y ago
        It was in 2006 which is way before AI started seeing it's recent progress. And it was a conference full of old fashioned AI people.
wskinner 10y ago
I wouldn't place too much stake in predictions made by experts. There is evidence that these may be far less accurate than we'd like to think, even for very near events. See http://press.princeton.edu/titles/7959.html http://press.princeton.edu/titles/7959.html. This research was also discussed in Thinking Fast and Slow.
- Houshalter 10y ago
  Individual experts may make very inaccurate predictions, sure. But averaged together you get the wisdom of crowds effect.
  - wskinner 10y ago
    Only if those experts' individual inaccuracies are uncorrelated - which for public intellectuals, they will probably always be. See https://en.wikipedia.org/wiki/The_Wisdom_of_Crowds#Failures_of_crowd_intelligence https://en.wikipedia.org/wiki/The_Wisdom_of_Crowds#Failures_...
hectorperez 10y ago
In the risks we should take into account problems caused by mass unemployment (caused by AI). By the way, there is a list of who does and who does not think there is a serious risk for society (of any type) within the next 50 years: https://goo.gl/Oe7S6e https://goo.gl/Oe7S6e
unprepare 10y ago
>Who knows where it will be in 40 years. Look where computer technology was 40 years ago. The Commodore PET, Apple II, and TRS-80 all came out exactly 40 years ago (1977)[1] [1]https://en.wikipedia.org/wiki/1977_in_science#Computer_science https://en.wikipedia.org/wiki/1977_in_science#Computer_scien...

j2kun 10y ago

Is there an established, rigorous definition of "high level machine intelligence"? Or even a concrete list of sufficient criteria? Every single discussion I've encountered leaves the notion undefined, or just "I'll know it when I see it"

Houshalter 10y ago
Artificial General Intelligence is roughly defined as a machine that can do all the things a human can do. For example, an AI capable of doing AI research and programming computers, would be AGI. The Turing test (as Turing originally described it, not garbage like chatbot competitions) is the most widely accepted standard of AGI. Superintelligence goes well beyond that. A machine with cognitive abilities far beyond humans. I don't think such a machine is very unlikely even in the near future. It's unreasonable to believe that humans are the pinnacle of intelligence. We are just the very first intelligent creature to evolve. Our brains are heavily resource constrained by size and energy. And neurons are many orders of magnitude slower than transistors. And also far larger and less compact.
- rtx 10y ago
  So how will this intelligent thing come about.
  - Houshalter 10y ago
    Once we have AIs as smart as humans, they can do AI research as good or better than human researchers. And they can make AIs that are even better, which in turn can make even better AIs, and so on. Dumb evolution was able to create human-level intelligence with just random mutations and natural selection. Surely human engineers can do better. But in the worst case, we could reverse engineer the human brain.
    - evgen 10y ago
      > Once we have AIs as smart as humans Whether a true 'generalist' AI is possible in the forseeable future is debatable. > they can do AI research as good or better than human researchers. Now your AI is not just a 'generalist' but rather a specialist in AI. A very big leap of faith has occurred here. This also presumes that the AIs are even capable of effective invention and improvisation rather than mimicry and optimization (the only two features we have seen from the very best of cutting-edge ML work so far.) > And they can make AIs that are even better, which in turn can make even better AIs, and so on. All of which is predicated on the limiting factor being software and not hardware. If it is the latter then these postulated AIs are hitting the same brick wall as humans and thinking about the problem faster or harder does not magically make the necessary hardware appear.
      - Houshalter 10y ago
        >Whether a true 'generalist' AI is possible in the forseeable future is debatable. Sure, but see the survey I posted above. The rate of progress of AI is incredible. We will almost certainly be approaching human level in a few decades at most. >Now your AI is not just a 'generalist' but rather a specialist in AI. That's what general intelligence is. The ability to learn different specializations. AI researchers are not literally born as AI researchers and capable of nothing else. > This also presumes that the AIs are even capable of effective invention and improvisation rather than mimicry and optimization Why wouldn't they be? If they are generally intelligent and can do all the same tasks humans can do. Whats magical about invention that would prevent computers from ever doing it? >All of which is predicated on the limiting factor being software and not hardware. All the same arguments apply to hardware. Hardware has been improving exponentially for a much longer time than AI has. And I think hardware may already be close enough. Transistors are orders of magnitude faster and denser than biological synapses.
AbrahamParangi 10y ago
How about: "It'll know you when it sees you"
quadcore 10y ago
Maybe that's not the right question. What I understand from the article is that the threat is about being manipulated by computer algorithms. We don't really care the computers are actually intelligent and thinking or whatnot. What's bad is that we are manipulated by computers and they automates more and more areas of our lives each year.
make3 10y ago
futurism as a whole is 100% un-scientific
- Houshalter 10y ago
  And does being unscientific make a thing wrong? Saying "we won't have strong AI in our lifetimes" is an equally strong claim that is also unscientific. You can't say "we can't know" because what does that even mean? That the outcome is 50/50? That's still pretty decent odds for something so significant!
  - make3 10y ago
    As a professional scientist in AI and deep learning, I see futurists making wild predictions about AI that I wouldn't see anyone serious in the field coming remotely close to making. I don't know how it is with other futurism fields like genetics, space travel and energy generation, but I expect that it's a similar situation. All I'm trying to say is, let's please keep a cool head and a wholly skeptical approach about all this.
    - Houshalter 10y ago
      But the survey I posted that started this discussion was a survey of experts in AI. Note that no one is claiming that skynet will appear any day now. Just that we will very likely have human-level AI in a few decades. I don't think that's a terribly wild or speculative claim.
  - make3 10y ago
    also please note I didn't say "we won't have strong AI in our lifetimes". Let's work on the actual developments specific of science instead of what we dream it to be. We want people to think about deep learning as a sophisticated pattern analysis algorithm instead of thinking that Skynet is going to start a coup any day now.

vectorpush 10y ago

Consider that 41% of those surveyed conclude that machines will never be able to "simulate learning and every other aspect of human intelligence". The next 41% conclude it is possible within an unbounded "more than 50 years". It seems to me that the vast majority of experts are pretty skeptical.

Houshalter 10y ago
That's not 41%, that's 4.1%. When asked when there would be a 90% of AI, the median answer was 2075.
- vectorpush 10y ago
  No. It's definitely 41%. Look at page 6 of your link The earliest that machines will be able to simulate learning and every other aspect of human intelligence: Within 10 years 6 5% Between 11 and 25 years 3 2% Between 26 and 50 years 14 11% More than 50 years 50 41% Never 50 41% Totals 123 100%
  - Houshalter 10y ago
    That's in the prior work section. It's a much older survey of one specific conference.
    - gydfi 10y ago
      And was the other survey a comprehensive survey of everybody? The divergence in poll results seems to suggest we should put even less credence on polls here than usual.
      - Houshalter 10y ago
        It was in 2006 which is way before AI started seeing it's recent progress. And it was a conference full of old fashioned AI people.

wskinner 10y ago

I wouldn't place too much stake in predictions made by experts. There is evidence that these may be far less accurate than we'd like to think, even for very near events. See http://press.princeton.edu/titles/7959.html http://press.princeton.edu/titles/7959.html. This research was also discussed in Thinking Fast and Slow.

Houshalter 10y ago
Individual experts may make very inaccurate predictions, sure. But averaged together you get the wisdom of crowds effect.
- wskinner 10y ago
  Only if those experts' individual inaccuracies are uncorrelated - which for public intellectuals, they will probably always be. See https://en.wikipedia.org/wiki/The_Wisdom_of_Crowds#Failures_of_crowd_intelligence https://en.wikipedia.org/wiki/The_Wisdom_of_Crowds#Failures_...

hectorperez 10y ago

In the risks we should take into account problems caused by mass unemployment (caused by AI). By the way, there is a list of who does and who does not think there is a serious risk for society (of any type) within the next 50 years: https://goo.gl/Oe7S6e https://goo.gl/Oe7S6e

unprepare 10y ago

>Who knows where it will be in 40 years. Look where computer technology was 40 years ago. The Commodore PET, Apple II, and TRS-80 all came out exactly 40 years ago (1977)[1] [1]https://en.wikipedia.org/wiki/1977_in_science#Computer_science https://en.wikipedia.org/wiki/1977_in_science#Computer_scien...