4 ms·

Here are a few more excellent resources on the potential and the dangers of machine intelligence. Even if you don't expect to read Nick Bostrom's Superintellige

Here are a few more excellent resources on the potential and the dangers of machine intelligence. Even if you don't expect to read Nick Bostrom's Superintelligence - a deep, provocative, and thoughtful book, but also very verbose - the above links will give you an excellent primer on humanity's prospects if and when we develop a true general AI: [Wait But Why] The AI Revolution, Part 1: How and when we achieve machine intelligence, e.g. strong AI - http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html http://waitbutwhy.com/2015/01/artificial-intelligence-revolu... The AI Revolution, Part 2: The species-level immortality we can hope for, and the extinction we have to fear - http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-2.html http://waitbutwhy.com/2015/01/artificial-intelligence-revolu... [Resources on Friendly and Unfriendly AI] Unfriendly AI: What AI looks like if it does not act expressly in our interests; e.g., a universe tiled over with paper clips - http://wiki.lesswrong.com/wiki/Unfriendly_artificial_intelligence http://wiki.lesswrong.com/wiki/Unfriendly_artificial_intelli... Friendly AI: Strategies for designing an AI that respect human morals and metamorals, including what we'd want if we were as wise as a superintelligence (coherent extrapolated volition) - http://en.wikipedia.org/wiki/Friendly_artificial_intelligence http://en.wikipedia.org/wiki/Friendly_artificial_intelligenc...

MollyR 12y ago
thanks for these links. They seem extremely interesting.
- MollyR 12y ago
  Just curious, why the downvotes ? On the welcome page, it says it's okay to say thanks, am I missing some unspoken rule about Hackernews ?
  - deleted 12y ago
    [deleted]
AndrewKemendo 12y ago
Here is the thing about all of those, none of them give concrete step by step examples of how we would get to "unfriendly" or "friendly" AGI respectively. Now, you might say - well of course because we don't know how an AGI will be built yet. Right! That is exactly the point. At this point all of the MIRI (Former SIAI) staffers have stopped trying to explain paths to unfriendly AGI and have been just assuming them for their solutions. Just look at Yudkowski's work a few years ago on CEV and you'll see how any attempt at formalization falls apart. Superintelligence just takes from the starting point they seeded in Global Catastrophic Risks, but doesn't actually address it in any more detail IMO. "Our Final Invention" tried also to do this but failed as well. To be clear the AGI community understands thoroughly the risks inherent in AGI - I would venture to say far more-so than the non-researcher. So they are worth exploring, but at a certain point you have to say "We aren't sure if the atmosphere will catch on fire or not but lets try it anyway because it is the next logical step." At some point we need to have the hard conversation about what humans will do in a world where we are less relevant. Or just stop pursuing it altogether and put a moratorium on AGI research. A terrible terrible idea in my opinion.

MollyR 12y ago

thanks for these links. They seem extremely interesting.

MollyR 12y ago
Just curious, why the downvotes ? On the welcome page, it says it's okay to say thanks, am I missing some unspoken rule about Hackernews ?
- deleted 12y ago
  [deleted]

AndrewKemendo 12y ago

Here is the thing about all of those, none of them give concrete step by step examples of how we would get to "unfriendly" or "friendly" AGI respectively. Now, you might say - well of course because we don't know how an AGI will be built yet. Right! That is exactly the point. At this point all of the MIRI (Former SIAI) staffers have stopped trying to explain paths to unfriendly AGI and have been just assuming them for their solutions. Just look at Yudkowski's work a few years ago on CEV and you'll see how any attempt at formalization falls apart. Superintelligence just takes from the starting point they seeded in Global Catastrophic Risks, but doesn't actually address it in any more detail IMO. "Our Final Invention" tried also to do this but failed as well. To be clear the AGI community understands thoroughly the risks inherent in AGI - I would venture to say far more-so than the non-researcher. So they are worth exploring, but at a certain point you have to say "We aren't sure if the atmosphere will catch on fire or not but lets try it anyway because it is the next logical step." At some point we need to have the hard conversation about what humans will do in a world where we are less relevant. Or just stop pursuing it altogether and put a moratorium on AGI research. A terrible terrible idea in my opinion.