4 ms·

Here's a survey of AI experts: http://www.nickbostrom.com/papers/survey.pdf http://www.nickbostrom.com/papers/survey.pdf >We thus designed a brief question- na

Here's a survey of AI experts: http://www.nickbostrom.com/papers/survey.pdf http://www.nickbostrom.com/papers/survey.pdf >We thus designed a brief question- naire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that high- level machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity.

argonaut 11y ago
I see this survey cited a lot. I'd rather not get into a "my experts are better than your experts" argument (going on in the other thread here), but I really don't like Bostrom's survey. There are two really glaring issues: 1) Bostrom surveyed participants in conferences, not people presenting papers. Anyone can attend a conference if they buy a ticket, expert or not. 2) Bostrom decided to survey: the conferences "Philosophy and Theory of AI," "AGI," and "AGI-Impacts," all of them organized by himself or his colleague. He also emailed the mailing list of "Members of the Greek Association for Artificial Intelligence." Neither of these conferences are known in ML (or technical, for that matter). I have never heard of this association. Just about the only legitimate methodology was emailing the top 100 AI researchers according to Microsoft Academic search (only 29 responded). It would have much more weight to me if it were, say, a survey of people presenting papers at NIPS and ICML (the top 2 machine learning conferences). Maybe throw in ICLR, CVPR (top deep learning conference, top CV conference with ML focus). Because, like it or not, the only reason people fear super-intelligence is because of the advances being made in machine learning, and by extension, deep learning. Not because of advances (there are few) in classic AI.
narrator 11y ago
Asking about technology in 2075 is ridiculous. If you had asked people in 1916 about 1975, nobody would have gotten anything right.
- Houshalter 11y ago
  Perhaps, but that's what the comment I am replying to was doing. If we are going to be making predictions about the future, we might as well take a survey of experts instead of trusting the predictions of a single one.
erikpukinskis 11y ago
So they asked a group of people who don't know how to build a model of human intelligence to model the trajectory of their modeling of intelligence and when, once they succeed, the model will model something even more intelligent than that? And then they took the most popular guess. I'm sorry, I don't believe that gives us any information about the future whatsoever. That methodology is nonsense.
- Houshalter 11y ago
  No, that's not their methodology at all. They did a survey of AI experts and asked them for their predictions, and then took the median of those predictions. That's perfectly fair, and in fact far more likely to be correct than taking a random experts prediction. See the wisdom of crowds. The best way to do it would be to assume each expert is equally likely to be correct, and then estimate what the probability is from there. 50% of experts say there is a 50% chance it will be developed by 2045. Therefore 50% by 2045 is a good estimate. In fact if anything it's an underestimate, since many of the other 50% of experts will still assign some probability to it happening before 2045, increasing the odds. If you want me to, I can do a simulation with the data given and find out what the exact numbers should be. But the numbers in that abstract are unlikely to be too far off, and if anything are an underestimate. As for whether it's valid to take the opinions of experts, that's a different issue. But 1) the predictions of experts are more likely to be correct than random HN commentators. 2) the predictions of a group of experts are more likely to be correct than a single expert, which is what this thread is about. And 3) uncertainty doesn't mean you can default to the hypothesis that it won't happen. If nothing else a survey of predictions provides a good prior.

argonaut 11y ago

I see this survey cited a lot. I'd rather not get into a "my experts are better than your experts" argument (going on in the other thread here), but I really don't like Bostrom's survey. There are two really glaring issues: 1) Bostrom surveyed participants in conferences, not people presenting papers. Anyone can attend a conference if they buy a ticket, expert or not. 2) Bostrom decided to survey: the conferences "Philosophy and Theory of AI," "AGI," and "AGI-Impacts," all of them organized by himself or his colleague. He also emailed the mailing list of "Members of the Greek Association for Artificial Intelligence." Neither of these conferences are known in ML (or technical, for that matter). I have never heard of this association. Just about the only legitimate methodology was emailing the top 100 AI researchers according to Microsoft Academic search (only 29 responded). It would have much more weight to me if it were, say, a survey of people presenting papers at NIPS and ICML (the top 2 machine learning conferences). Maybe throw in ICLR, CVPR (top deep learning conference, top CV conference with ML focus). Because, like it or not, the only reason people fear super-intelligence is because of the advances being made in machine learning, and by extension, deep learning. Not because of advances (there are few) in classic AI.

narrator 11y ago

Asking about technology in 2075 is ridiculous. If you had asked people in 1916 about 1975, nobody would have gotten anything right.

Houshalter 11y ago
Perhaps, but that's what the comment I am replying to was doing. If we are going to be making predictions about the future, we might as well take a survey of experts instead of trusting the predictions of a single one.

erikpukinskis 11y ago

So they asked a group of people who don't know how to build a model of human intelligence to model the trajectory of their modeling of intelligence and when, once they succeed, the model will model something even more intelligent than that? And then they took the most popular guess. I'm sorry, I don't believe that gives us any information about the future whatsoever. That methodology is nonsense.

Houshalter 11y ago
No, that's not their methodology at all. They did a survey of AI experts and asked them for their predictions, and then took the median of those predictions. That's perfectly fair, and in fact far more likely to be correct than taking a random experts prediction. See the wisdom of crowds. The best way to do it would be to assume each expert is equally likely to be correct, and then estimate what the probability is from there. 50% of experts say there is a 50% chance it will be developed by 2045. Therefore 50% by 2045 is a good estimate. In fact if anything it's an underestimate, since many of the other 50% of experts will still assign some probability to it happening before 2045, increasing the odds. If you want me to, I can do a simulation with the data given and find out what the exact numbers should be. But the numbers in that abstract are unlikely to be too far off, and if anything are an underestimate. As for whether it's valid to take the opinions of experts, that's a different issue. But 1) the predictions of experts are more likely to be correct than random HN commentators. 2) the predictions of a group of experts are more likely to be correct than a single expert, which is what this thread is about. And 3) uncertainty doesn't mean you can default to the hypothesis that it won't happen. If nothing else a survey of predictions provides a good prior.