4 ms·

The purpose of the profile isn't to argue a risk exists. We largely defer to the people we take to be experts on the issue, especially Nick Bostrom. We think he

The purpose of the profile isn't to argue a risk exists. We largely defer to the people we take to be experts on the issue, especially Nick Bostrom. We think he presents compelling arguments in Superintelligence, and although it's hard to say anything decisive in this area, if you think there's even modest uncertainty about whether AGI will be good or bad, it's worth doing more research into the risks. If you haven't read Bostrom's book yet, I'd really recommend it. http://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/1501227742 http://www.amazon.com/Superintelligence-Dangers-Strategies-N...

manish_gill 11y ago
I read this book. Got about 4 chapters in before I had to give up at the sheer ridiculousness of the whole thing. The problem with this entire line of reasoning is that at this point it is nothing more an a thought experiment. Many of the key underlying assumptions that are required for Artificial General Intelligence simply have not been realised, and while Weak AI is progressing at a strong rate, we are hardly anywhere close to a place where we should start worrying about all this stuff. It's about as likely as a meteor hitting the planet and wiping out the entire human life. Possible? Sure. Should I be panicking about this right now? Nah. The book is nonsense. I'll start paying attention when someone who has real experience in the field of AI research (and I'm not talking charlatans like Yudkowsky here), but someone like say Norvig come out and say it's a reasonable concern today.
- BenjaminTodd 11y ago
  The expert consensus says 10% chance of human-level AI in 10 years: http://www.givewell.org/labs/causes/ai-risk/ai-timelines http://www.givewell.org/labs/causes/ai-risk/ai-timelines Many computer science professors have publicly said they think AI poses significant risks. There's a list here: http://slatestarcodex.com/2015/05/22/ai-researchers-on-ai-risk/ http://slatestarcodex.com/2015/05/22/ai-researchers-on-ai-ri... Also see this open letter, signed by hundreds of experts: http://futureoflife.org/ai-open-letter/ http://futureoflife.org/ai-open-letter/
  - manish_gill 11y ago
    Your last link, the open letter, says nothing about human or > human level AI. Just "robust and beneficial usage" of AI. In all likelihood that means the current AI technology and the letter is aimed at (I'm assuming) people who are trying to use these techniques in things such as modern weapon systems. While that is of course a concern, it's not the same as the concern for a Skynet like scenario. Some experts in the second link you gave are concerned, sure. But I can probably find an equal number who dismiss it as well. There isn't a clear consensus over AGI. I still remain skeptical. Same with your first link, which tries to "forecast" AGI. People can't forecast next month's weather correctly, so forgive me for not believing in a 10% chance in 10 years. Actually, I take back my appeal to authority argument in its entirety, because I just remembered the first thing I saw in my AI class was a video of experts claiming the exact same thing. The video was from the 50s. EDIT: Found it: https://www.youtube.com/watch?v=rlBjhD1oGQg https://www.youtube.com/watch?v=rlBjhD1oGQg
    - BenjaminTodd 11y ago
      The letter isn't (just) about modern weapon systems. It was put together by this group: http://futureoflife.org/ai-news/ http://futureoflife.org/ai-news/ Also, no-one is worried about a skynet scenario. The worrying scenario is just any powerful system that optimises for something that's different from what humans want. Second, the point is that even uncertainty is enough for action. For AI to not be a problem, you'd need to be very confident that it'll occur a long way in the future, and that there's nothing we can do until it's closer. As you've said, we don't have confidence in the timeline. We have large uncertainty. And that's more reason for action, especially research. Consider analogously: "We've got no idea what the chance of run-away climate change is, so we shouldn't do anything about it." Seems like a bad argument to me.
      - andreyf 11y ago
        Also, no-one is worried about a skynet scenario. The worrying scenario is just any powerful system that optimises for something that's different from what humans want. So Transcendence in leu of The Terminator. It's still hollywood fiction.
- derefr 11y ago
  > we are hardly anywhere close to a place where we should start worrying about all this stuff Here's a comparison: wouldn't it be great if we had started thinking about climate change way back at the beginning of the industrial revolution, before we decided to create tons of open-air coal plants? There are reasons to solve problems before they're problems.
  - manish_gill 11y ago
    While I agree that there should most definitely be general research in the area, and it is of course something worth exploring, what I don't agree with is the hype that is being built around this. Hollywood has played its part of course, but people who should really know better seem to be under the impression that as soon as we achieve AGI (which is a big /if/), we're doomed immediately. They ignore that all this is still academic and while industry practices follow sooner than is usual these days, it's not like one day a scientist creates AI and the next day, Skynet launches Nuclear Missiles. So is it worth exploring? Sure. But I'm not going to be concerned about something for which there is zero evidence to support it. There are no reasons to solve problems before there is foolproof evidence that they are indeed problems.
deleted 11y ago
[deleted]

manish_gill 11y ago

I read this book. Got about 4 chapters in before I had to give up at the sheer ridiculousness of the whole thing. The problem with this entire line of reasoning is that at this point it is nothing more an a thought experiment. Many of the key underlying assumptions that are required for Artificial General Intelligence simply have not been realised, and while Weak AI is progressing at a strong rate, we are hardly anywhere close to a place where we should start worrying about all this stuff. It's about as likely as a meteor hitting the planet and wiping out the entire human life. Possible? Sure. Should I be panicking about this right now? Nah. The book is nonsense. I'll start paying attention when someone who has real experience in the field of AI research (and I'm not talking charlatans like Yudkowsky here), but someone like say Norvig come out and say it's a reasonable concern today.

BenjaminTodd 11y ago
The expert consensus says 10% chance of human-level AI in 10 years: http://www.givewell.org/labs/causes/ai-risk/ai-timelines http://www.givewell.org/labs/causes/ai-risk/ai-timelines Many computer science professors have publicly said they think AI poses significant risks. There's a list here: http://slatestarcodex.com/2015/05/22/ai-researchers-on-ai-risk/ http://slatestarcodex.com/2015/05/22/ai-researchers-on-ai-ri... Also see this open letter, signed by hundreds of experts: http://futureoflife.org/ai-open-letter/ http://futureoflife.org/ai-open-letter/
- manish_gill 11y ago
  Your last link, the open letter, says nothing about human or > human level AI. Just "robust and beneficial usage" of AI. In all likelihood that means the current AI technology and the letter is aimed at (I'm assuming) people who are trying to use these techniques in things such as modern weapon systems. While that is of course a concern, it's not the same as the concern for a Skynet like scenario. Some experts in the second link you gave are concerned, sure. But I can probably find an equal number who dismiss it as well. There isn't a clear consensus over AGI. I still remain skeptical. Same with your first link, which tries to "forecast" AGI. People can't forecast next month's weather correctly, so forgive me for not believing in a 10% chance in 10 years. Actually, I take back my appeal to authority argument in its entirety, because I just remembered the first thing I saw in my AI class was a video of experts claiming the exact same thing. The video was from the 50s. EDIT: Found it: https://www.youtube.com/watch?v=rlBjhD1oGQg https://www.youtube.com/watch?v=rlBjhD1oGQg
  - BenjaminTodd 11y ago
    The letter isn't (just) about modern weapon systems. It was put together by this group: http://futureoflife.org/ai-news/ http://futureoflife.org/ai-news/ Also, no-one is worried about a skynet scenario. The worrying scenario is just any powerful system that optimises for something that's different from what humans want. Second, the point is that even uncertainty is enough for action. For AI to not be a problem, you'd need to be very confident that it'll occur a long way in the future, and that there's nothing we can do until it's closer. As you've said, we don't have confidence in the timeline. We have large uncertainty. And that's more reason for action, especially research. Consider analogously: "We've got no idea what the chance of run-away climate change is, so we shouldn't do anything about it." Seems like a bad argument to me.
    - andreyf 11y ago
      Also, no-one is worried about a skynet scenario. The worrying scenario is just any powerful system that optimises for something that's different from what humans want. So Transcendence in leu of The Terminator. It's still hollywood fiction.
derefr 11y ago
> we are hardly anywhere close to a place where we should start worrying about all this stuff Here's a comparison: wouldn't it be great if we had started thinking about climate change way back at the beginning of the industrial revolution, before we decided to create tons of open-air coal plants? There are reasons to solve problems before they're problems.
- manish_gill 11y ago
  While I agree that there should most definitely be general research in the area, and it is of course something worth exploring, what I don't agree with is the hype that is being built around this. Hollywood has played its part of course, but people who should really know better seem to be under the impression that as soon as we achieve AGI (which is a big /if/), we're doomed immediately. They ignore that all this is still academic and while industry practices follow sooner than is usual these days, it's not like one day a scientist creates AI and the next day, Skynet launches Nuclear Missiles. So is it worth exploring? Sure. But I'm not going to be concerned about something for which there is zero evidence to support it. There are no reasons to solve problems before there is foolproof evidence that they are indeed problems.

deleted 11y ago

[deleted]