5 ms·

I have yet to meet a serious AI researcher who worries about AI ending the human race. At every AI conference I've been to lately, some guy would inevitably ask

I have yet to meet a serious AI researcher who worries about AI ending the human race. At every AI conference I've been to lately, some guy would inevitably ask the same question to the speaker: "do you think we should be concerned by superhuman AI?". The answer was always the same, for instance from Andrew Ng at the Deep Learning Summit a few weeks ago: "dude, stop it. That's such a distraction". > One of my top 4 favorite explanations for the Fermi paradox is that biological intelligence always eventually creates machine intelligence, which wipes out biological life and then for some reason decides to makes itself undetectable. Since you are so knowledgeable about the unknowable, may I ask, Sam, do you think angels are male, or are they female? It has been a long standing question among the sort of people who like to predict the coming end of the world. Seriously, billionaires and pundits warning us about our impending AI doom is is such a distraction. Does AI have its dangers? Yes. And nobody talks about them. The danger of AI is that is will put increasing power in the hands of those who have the data and the know-how, i.e. large corporations and governments. The power of mining usefully the troves of data they have on every citizen, both on a micro level and a macro level, to understand what people are doing, what they are thinking, and what they are going to do next. And ultimately, to control what they think (to give you an idea, Facebook can already influence your mood by selecting what goes into your newsfeed). First comes intelligence, then prediction, and finally, control. Yes, AI is something we should probably be worried about, in the same way that we should have discussed the privacy implications of the Internet years before the NSA files. But a terminator-like end of the world scenario is not a concern. If you are clueless about a topic, please refrain from making grand statements about its future. A great quote from Neil Gershenfeld: "The history of technology advancing has been one of sigmoids that begin as exponentials."

mej10 12y ago
How about Shane Legg (One of the cofounders of DeepMind)? http://lesswrong.com/lw/691/qa_with_shane_legg_on_risks_from_ai/ http://lesswrong.com/lw/691/qa_with_shane_legg_on_risks_from... Quote: Q6: Do possible risks from AI outweigh other possible existential risks, e.g. risks associated with the possibility of advanced nanotechnology? Shane Legg: It's my number 1 risk for this century, with an engineered biological pathogen coming a close second (though I know little about the latter).
- fchollet 12y ago
  Shane Legg is known for being a co-founder of DeepMind, in a business role (as I understand). He's a complete nobody as a researcher (is he even an AI researcher? I would be surprised). The big names of deep learning have all taken a vocal stance against the recent end-of-the-world punditry (most notably Yann LeCun and Andrew Ng). Also notable: roboticist Rodney Brooks http://www.rethinkrobotics.com/artificial-intelligence-tool-threat/ http://www.rethinkrobotics.com/artificial-intelligence-tool-...
  - oskarth 12y ago
    Since you seem to be well-versed in this world, do you know what reputation Nick Bostrom has in these circles?
    - seiji 12y ago
      There's a very fine line between AI futurist and best-guess scifi writer. Most "AI thought leaders" are scifi writers, not technical researchers. They take preconditions, generate a story, think how it could happen given plausible technology, then market that as soon-to-be-fact. It's an entertaining society and endlessly fun to read, but still complete fiction based on internal brain states of individuals and not necessarily based on real world interactions. Also see: Eliezer Yudkowsky — great writer, fun to read, but largely scifi thought experiments masquerading as research.
    - jeffreyrogers 12y ago
      The only times I've heard him mentioned the impression was negative and that he didn't understand any of the actual science. People hear "machine learning" and they think it is about machines that know how to think. Machine learning is actually just optimization of high dimensional functions. If this language were used it wouldn't sound as sexy, but no one would think machines are going to take over the world. AI isn't magic. It's really just clever search techniques and mathematical optimization.
      - snth 12y ago
jey 12y ago
Stuart Russell, AI professor at UC Berkeley and co-author of the Artificial Intelligence: A Modern Approach textbook, cares: http://edge.org/conversation/the-myth-of-ai#26015 http://edge.org/conversation/the-myth-of-ai#26015
- deleted 12y ago
  [deleted]
- fchollet 12y ago
  His thoughts seem quite sensible, although extremely abstract and theoretical. This is lightyears away from the Elon Musk / Bill Gates / etc. fear-mongering.
  - mej10 12y ago
    So do you take his thoughts seriously or not? Do you now think AI researchers are engaging in unethical behavior since they don't care about AI safety?
    - deleted 12y ago
      [deleted]
deleted 12y ago
[deleted]
Tossrock 12y ago
I have a feeling this is a leakage of Eliezer Yudkowsky's opinions into tech entrepreneur circles via his friendship with Peter Thiel.
- AndrewKemendo 12y ago
  This is exactly what it is, which is so terribly terribly disheartening as Yudkowski is at best a marginal sci-fi writer with and outsized ego and some math chops. He has an almost cult like following over at less wrong. His writings don't do anything for fundamental AI research but just handwave a lot of philosophical arguments.
- fchollet 12y ago
  I think you nailed it. "AI doom" is a powerful psychological trope for people in technology, and much like the hypothetical rogue super-AIs, it has gone out of control.
convexfunction 12y ago
Andrew, in that case, is definitely right in that long-term AI safety has almost nothing to do with near-term AI implementation. More generally, when I see people assert that these people don't know what they're talking about, it pretty much always seems like a case of reference class confusion. People seem to expect that AI safety research must look exactly like AI implementation research, otherwise it's illegitimate. (Kind of like how biology research must look exactly like chemistry research, otherwise it's illegitimate.) This is a game theory problem; it might be informed by knowledge of machine learning, like how a broad understanding of chemistry helps in much biology research, but they're not the same level of abstraction; insisting that anyone who wants to study a problem that kind of touches on your own research interests must do it in exactly the same way you do it and focus on exactly the same things, otherwise they're hacks and crackpots, reeks of narrow-mindedness.
- oskarth 12y ago
  This is a good point I completely missed. Reminds me of Karl Popper qua philosopher (of science) : science.
edanm 12y ago
What would it take to convince you this is something worth worrying about? Being very interested in this field myself for a few years, I know plenty of people who do worry about AI. But since other people have already mentioned some of them in the comments and been dismissed, I want to make sure we're on the same page: what do you consider to be convincing evidence that it's an issue worth thinking about?
gojomo 12y ago
How about Jeremy Howard, former President of crowd-ML-competition site Kaggle and now founder of ML-medicine company Enlitic? http://www.reddit.com/r/Futurology/comments/2p6k20/im_jeremy_howard_enlitic_ceo_kaggle_past/cmtvspz http://www.reddit.com/r/Futurology/comments/2p6k20/im_jeremy... The opinions of leading-edge researchers are valuable but not necessarily dispositive. Often, it benefits a researcher to be oblivious to the larger or darker implications of their work. Boundless optimism and a maniacal focus on only-what-is-knowable-right-now delivers publishable/actionable results, all the other 'speculation' is genuinely a "distraction" for them. That doesn't mean it's a distraction for us. Often the speculators and entrepreneurs ("billionaires and pundits") are more talented than researchers at projecting trends and social-economic interactions.
upquark 12y ago
> I have yet to meet a serious AI researcher who worries about AI ending the human race. As an industry practitioner of machine learning / data science, I believe AGI poses a genuine risk to humanity. Having said that, what I do for a living is of little relevance. People are pretty terrible at predicting the future (see late 19th / early 20th century predictions of year 2000, with food replaced by pills, etc). Unless someone has put enough thought and research into it, their predictions about the future of civilization are likely to be worthless regardless of their academic credentials.
Sakes 12y ago
I don't understand. > Since you are so knowledgeable about the unknowable If knowing the unknown was a requirement to exploring it through playful thought, forming opinions and refining them over time through discussion, reflection and if possible testing, well... we'd still be chunking rocks. I think it is unfair to compare expressing concerns about the implications of AI to debating the gender of angels. No one is trying to build angels... > for instance from Andrew Ng at the Deep Learning Summit a few weeks ago: "dude, stop it. That's such a distraction". Ya, this makes sense in the context of the summit. But, applying it to a guy's blog? It is a blog... > Yes, AI is something we should probably be worried about. Agreed! > "The history of technology advancing has been one of sigmoids that begin as exponentials." Sure, but will we be irrelevant before the slope changes direction? Anyways, I share your fear of advancements in technology being used to secure entrenched power structures. And yes, if it happened it would come before the singularity. But I just can't imagine how anyone wouldn't be afraid to be part of a species that had stopped evolving while a new competing "life" form was evolving at rates that known history has never seen.
rl3 12y ago
>Since you are so knowledgeable about the unknowable, may I ask, Sam, do you think angels are male, or are they female? How condescending.
Houshalter 12y ago
The concept of singularities and intelligence explosions and unfriendly AI is only starting to become mainstream, even among AI researchers. When asked about the dangers of AI, they mostly talk about the near future and their current work. The dangers of AI do not come from current work in the near future, but human level AIs decades from now. People are talking about two very different things in the same conversations using the same words. As for what expert opinion actually is on this subject, there is a good survey here: http://www.nickbostrom.com/papers/survey.pdf http://www.nickbostrom.com/papers/survey.pdf We thus designed a brief questionnaire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that high-level machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity.
azakai 12y ago
> I have yet to meet a serious AI researcher who worries about AI ending the human race. At every AI conference I've been to lately, [the answer was always] "dude, stop it. That's such a distraction". Sure, it's a distraction at AI conferences. It's a distraction from the daily, monthly, yearly even, work of AI. AI researchers have a million problems to solve in the near term, which are hard and interesting. Speculating about the farther future is perhaps fun to do, but doesn't get papers published, and doesn't help with any of the practical problems AI researchers have today. That doesn't mean it isn't worth thinking about the long-term dangers of AI. Someone should, as while the danger might be unlikely to happen soon, but if it does eventually happen, it could end us. The same is true of a human-engineered pandemic virus - hopefully unlikely, but someone should be preparing us. We have to handle some worst-cases. AI researchers are not necessarily the people interested to think about the long-term dangers of AI, because they focus on the field as it stands today, not where it could be in a generation. Of course, their input is crucial to making specific guesses about where the field is going. But calculating the dangers of AI is not just a matter for AI researchers, just like the benefits of medicine is not just a purely medical issue (the cost and availability of medicine play a huge part in how effective medicine is, at a societal level, and are things not under the control of doctors).
richardw 12y ago
Argue the points, not the man. "The danger of AI is that is will put increasing power in the hands of those who have the data and the know-how, i.e. large corporations and governments." This assumes AI that continues to serve corporations and governments without question, despite increasing approximations of intelligence. Can you guarantee that? It's like tasking monkeys to create rules that humans can't escape from. I'd bet on the humans. We can't even create financial rules that we can't evade. The risk in any complex system is emergent, unintended consequence.
nrao123 12y ago
Here is a list of some famous people concerned about AI: http://pansop.com/1002/ http://pansop.com/1002/ This includes Shane Legg & Demis Hassabis who co founded Deepmind, an AI startup

mej10 12y ago

How about Shane Legg (One of the cofounders of DeepMind)? http://lesswrong.com/lw/691/qa_with_shane_legg_on_risks_from_ai/ http://lesswrong.com/lw/691/qa_with_shane_legg_on_risks_from... Quote: Q6: Do possible risks from AI outweigh other possible existential risks, e.g. risks associated with the possibility of advanced nanotechnology? Shane Legg: It's my number 1 risk for this century, with an engineered biological pathogen coming a close second (though I know little about the latter).

fchollet 12y ago
Shane Legg is known for being a co-founder of DeepMind, in a business role (as I understand). He's a complete nobody as a researcher (is he even an AI researcher? I would be surprised). The big names of deep learning have all taken a vocal stance against the recent end-of-the-world punditry (most notably Yann LeCun and Andrew Ng). Also notable: roboticist Rodney Brooks http://www.rethinkrobotics.com/artificial-intelligence-tool-threat/ http://www.rethinkrobotics.com/artificial-intelligence-tool-...
- oskarth 12y ago
  Since you seem to be well-versed in this world, do you know what reputation Nick Bostrom has in these circles?
  - seiji 12y ago
    There's a very fine line between AI futurist and best-guess scifi writer. Most "AI thought leaders" are scifi writers, not technical researchers. They take preconditions, generate a story, think how it could happen given plausible technology, then market that as soon-to-be-fact. It's an entertaining society and endlessly fun to read, but still complete fiction based on internal brain states of individuals and not necessarily based on real world interactions. Also see: Eliezer Yudkowsky — great writer, fun to read, but largely scifi thought experiments masquerading as research.
  - jeffreyrogers 12y ago
    The only times I've heard him mentioned the impression was negative and that he didn't understand any of the actual science. People hear "machine learning" and they think it is about machines that know how to think. Machine learning is actually just optimization of high dimensional functions. If this language were used it wouldn't sound as sexy, but no one would think machines are going to take over the world. AI isn't magic. It's really just clever search techniques and mathematical optimization.
    - snth 12y ago

jey 12y ago

Stuart Russell, AI professor at UC Berkeley and co-author of the Artificial Intelligence: A Modern Approach textbook, cares: http://edge.org/conversation/the-myth-of-ai#26015 http://edge.org/conversation/the-myth-of-ai#26015

deleted 12y ago
[deleted]
fchollet 12y ago
His thoughts seem quite sensible, although extremely abstract and theoretical. This is lightyears away from the Elon Musk / Bill Gates / etc. fear-mongering.
- mej10 12y ago
  So do you take his thoughts seriously or not? Do you now think AI researchers are engaging in unethical behavior since they don't care about AI safety?
  - deleted 12y ago
    [deleted]

deleted 12y ago

[deleted]

Tossrock 12y ago

I have a feeling this is a leakage of Eliezer Yudkowsky's opinions into tech entrepreneur circles via his friendship with Peter Thiel.

AndrewKemendo 12y ago
This is exactly what it is, which is so terribly terribly disheartening as Yudkowski is at best a marginal sci-fi writer with and outsized ego and some math chops. He has an almost cult like following over at less wrong. His writings don't do anything for fundamental AI research but just handwave a lot of philosophical arguments.
fchollet 12y ago
I think you nailed it. "AI doom" is a powerful psychological trope for people in technology, and much like the hypothetical rogue super-AIs, it has gone out of control.

convexfunction 12y ago

Andrew, in that case, is definitely right in that long-term AI safety has almost nothing to do with near-term AI implementation. More generally, when I see people assert that these people don't know what they're talking about, it pretty much always seems like a case of reference class confusion. People seem to expect that AI safety research must look exactly like AI implementation research, otherwise it's illegitimate. (Kind of like how biology research must look exactly like chemistry research, otherwise it's illegitimate.) This is a game theory problem; it might be informed by knowledge of machine learning, like how a broad understanding of chemistry helps in much biology research, but they're not the same level of abstraction; insisting that anyone who wants to study a problem that kind of touches on your own research interests must do it in exactly the same way you do it and focus on exactly the same things, otherwise they're hacks and crackpots, reeks of narrow-mindedness.

oskarth 12y ago
This is a good point I completely missed. Reminds me of Karl Popper qua philosopher (of science) : science.

edanm 12y ago

What would it take to convince you this is something worth worrying about? Being very interested in this field myself for a few years, I know plenty of people who do worry about AI. But since other people have already mentioned some of them in the comments and been dismissed, I want to make sure we're on the same page: what do you consider to be convincing evidence that it's an issue worth thinking about?

gojomo 12y ago

How about Jeremy Howard, former President of crowd-ML-competition site Kaggle and now founder of ML-medicine company Enlitic? http://www.reddit.com/r/Futurology/comments/2p6k20/im_jeremy_howard_enlitic_ceo_kaggle_past/cmtvspz http://www.reddit.com/r/Futurology/comments/2p6k20/im_jeremy... The opinions of leading-edge researchers are valuable but not necessarily dispositive. Often, it benefits a researcher to be oblivious to the larger or darker implications of their work. Boundless optimism and a maniacal focus on only-what-is-knowable-right-now delivers publishable/actionable results, all the other 'speculation' is genuinely a "distraction" for them. That doesn't mean it's a distraction for us. Often the speculators and entrepreneurs ("billionaires and pundits") are more talented than researchers at projecting trends and social-economic interactions.

upquark 12y ago

> I have yet to meet a serious AI researcher who worries about AI ending the human race. As an industry practitioner of machine learning / data science, I believe AGI poses a genuine risk to humanity. Having said that, what I do for a living is of little relevance. People are pretty terrible at predicting the future (see late 19th / early 20th century predictions of year 2000, with food replaced by pills, etc). Unless someone has put enough thought and research into it, their predictions about the future of civilization are likely to be worthless regardless of their academic credentials.

Sakes 12y ago

I don't understand. > Since you are so knowledgeable about the unknowable If knowing the unknown was a requirement to exploring it through playful thought, forming opinions and refining them over time through discussion, reflection and if possible testing, well... we'd still be chunking rocks. I think it is unfair to compare expressing concerns about the implications of AI to debating the gender of angels. No one is trying to build angels... > for instance from Andrew Ng at the Deep Learning Summit a few weeks ago: "dude, stop it. That's such a distraction". Ya, this makes sense in the context of the summit. But, applying it to a guy's blog? It is a blog... > Yes, AI is something we should probably be worried about. Agreed! > "The history of technology advancing has been one of sigmoids that begin as exponentials." Sure, but will we be irrelevant before the slope changes direction? Anyways, I share your fear of advancements in technology being used to secure entrenched power structures. And yes, if it happened it would come before the singularity. But I just can't imagine how anyone wouldn't be afraid to be part of a species that had stopped evolving while a new competing "life" form was evolving at rates that known history has never seen.

rl3 12y ago

>Since you are so knowledgeable about the unknowable, may I ask, Sam, do you think angels are male, or are they female? How condescending.

Houshalter 12y ago

The concept of singularities and intelligence explosions and unfriendly AI is only starting to become mainstream, even among AI researchers. When asked about the dangers of AI, they mostly talk about the near future and their current work. The dangers of AI do not come from current work in the near future, but human level AIs decades from now. People are talking about two very different things in the same conversations using the same words. As for what expert opinion actually is on this subject, there is a good survey here: http://www.nickbostrom.com/papers/survey.pdf http://www.nickbostrom.com/papers/survey.pdf We thus designed a brief questionnaire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that high-level machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity.

azakai 12y ago

> I have yet to meet a serious AI researcher who worries about AI ending the human race. At every AI conference I've been to lately, [the answer was always] "dude, stop it. That's such a distraction". Sure, it's a distraction at AI conferences. It's a distraction from the daily, monthly, yearly even, work of AI. AI researchers have a million problems to solve in the near term, which are hard and interesting. Speculating about the farther future is perhaps fun to do, but doesn't get papers published, and doesn't help with any of the practical problems AI researchers have today. That doesn't mean it isn't worth thinking about the long-term dangers of AI. Someone should, as while the danger might be unlikely to happen soon, but if it does eventually happen, it could end us. The same is true of a human-engineered pandemic virus - hopefully unlikely, but someone should be preparing us. We have to handle some worst-cases. AI researchers are not necessarily the people interested to think about the long-term dangers of AI, because they focus on the field as it stands today, not where it could be in a generation. Of course, their input is crucial to making specific guesses about where the field is going. But calculating the dangers of AI is not just a matter for AI researchers, just like the benefits of medicine is not just a purely medical issue (the cost and availability of medicine play a huge part in how effective medicine is, at a societal level, and are things not under the control of doctors).

richardw 12y ago

Argue the points, not the man. "The danger of AI is that is will put increasing power in the hands of those who have the data and the know-how, i.e. large corporations and governments." This assumes AI that continues to serve corporations and governments without question, despite increasing approximations of intelligence. Can you guarantee that? It's like tasking monkeys to create rules that humans can't escape from. I'd bet on the humans. We can't even create financial rules that we can't evade. The risk in any complex system is emergent, unintended consequence.

nrao123 12y ago

Here is a list of some famous people concerned about AI: http://pansop.com/1002/ http://pansop.com/1002/ This includes Shane Legg & Demis Hassabis who co founded Deepmind, an AI startup