4 ms·

Let's not kid ourselves, there is simply no way to ensure the promise outweighs the perils. I'm talking about superintelligence here, not artificial general in

Let's not kid ourselves, there is simply no way to ensure the promise outweighs the perils. I'm talking about superintelligence here, not artificial general intelligence, although it's almost certain superintelligence will arrive shortly after AGI. Anyone who is not scared shitless of what's to come simply does not truly understand what AI is capable of. If you're one of those people, do not compare AI to ANYTHING that you are familiar with today in regards to computers/software. Superintelligent AI will be like a god that knows your inner thoughts, desires, hopes and dreams as well as nearly everyone else on earth. It will manipulate people with ease - it's not even an issue. Whatever it wants to accomplish, it will, with or without human approval.

jimrandomh 11y ago
While I agree that we are not currently ready for the arrival of superintelligence, I think it's worth emphasizing that there is time to prepare, that there is useful preparation to be done which may get things to a point where it will be safe (or at least marginally safer), and that if you are feeling worried, it may be best to channel that feeling into useful action and get involved with the research! Good places to start are the Machine Intelligence Research Institute's Research Guide (https://intelligence.org/research-guide/ https://intelligence.org/research-guide/), which summarizes the aspects of the problem they've worked on and think are important; and the Future of Life Institute's Research Priorities (http://futureoflife.org/static/data/documents/research_priorities.pdf http://futureoflife.org/static/data/documents/research_prior...), particularly Section 3.
- sixQuarks 11y ago
  Don't get me wrong, I'm certainly not against trying to prepare against it - perhaps we can discover a breakthrough that does defend against AI (although I'm very doubtful in the long-term). It is worth a serious try though.
xamdam 11y ago
I largely agree with you (have been supporting intelligence.org before the topic was cool) but I think if we don't achieve human-friendly AI it's on us; a real failure rather than just succumbing to circumstances (like getting blown away by a supernova). Right now we have control and we can release AGI only if we have very high confidence that it aligns well with human value. The analogy that is somewhat relevant is that basic drives can be controlled even if AI's intellectual power leaves us in the dust, like we know that most normal humans will care about their children (and senile parents) despite being cognitively superior. We have to fight denial (many AI researchers are IMO engaged in it) apathy and fatalism, and in the end with patience we can have something great.
- sixQuarks 11y ago
  Interesting info came out during interview with Elon Musk recently: "— He is very, very concerned about AI. I quoted him in my posts on AI saying that he fears that by working to bring about Superintelligent AI (ASI), we’re “summoning the demon,” but I didn’t know how much he thought about the topic. He cited AI safety as one of the three things he thinks about most—the other two being sustainable energy and becoming a multi-planet species, i.e. Tesla and SpaceX. Musk is a smart motherfucker, and he knows a ton about AI, and his sincere concern about this makes me scared." from: http://waitbutwhy.com/2015/05/elon-musk-the-worlds-raddest-man.html http://waitbutwhy.com/2015/05/elon-musk-the-worlds-raddest-m...
  - xamdam 11y ago
    Yes, absolutely there is a reason for concern. I now volunteer for FLI (the organization Musk funded; we deal with both AI and other x-risks, synthetic bio etc). From a somewhat insider view I think there is a lot we can do, and time might be on our side (AGI is a pretty hard problem). I'm cautiously optimistic that we (rational and altruistic humans) can make a difference.
    - sixQuarks 11y ago
      keep up the good fight! Thank you for your efforts

jimrandomh 11y ago

While I agree that we are not currently ready for the arrival of superintelligence, I think it's worth emphasizing that there is time to prepare, that there is useful preparation to be done which may get things to a point where it will be safe (or at least marginally safer), and that if you are feeling worried, it may be best to channel that feeling into useful action and get involved with the research! Good places to start are the Machine Intelligence Research Institute's Research Guide (https://intelligence.org/research-guide/ https://intelligence.org/research-guide/), which summarizes the aspects of the problem they've worked on and think are important; and the Future of Life Institute's Research Priorities (http://futureoflife.org/static/data/documents/research_priorities.pdf http://futureoflife.org/static/data/documents/research_prior...), particularly Section 3.

sixQuarks 11y ago
Don't get me wrong, I'm certainly not against trying to prepare against it - perhaps we can discover a breakthrough that does defend against AI (although I'm very doubtful in the long-term). It is worth a serious try though.

xamdam 11y ago

I largely agree with you (have been supporting intelligence.org before the topic was cool) but I think if we don't achieve human-friendly AI it's on us; a real failure rather than just succumbing to circumstances (like getting blown away by a supernova). Right now we have control and we can release AGI only if we have very high confidence that it aligns well with human value. The analogy that is somewhat relevant is that basic drives can be controlled even if AI's intellectual power leaves us in the dust, like we know that most normal humans will care about their children (and senile parents) despite being cognitively superior. We have to fight denial (many AI researchers are IMO engaged in it) apathy and fatalism, and in the end with patience we can have something great.

sixQuarks 11y ago
Interesting info came out during interview with Elon Musk recently: "— He is very, very concerned about AI. I quoted him in my posts on AI saying that he fears that by working to bring about Superintelligent AI (ASI), we’re “summoning the demon,” but I didn’t know how much he thought about the topic. He cited AI safety as one of the three things he thinks about most—the other two being sustainable energy and becoming a multi-planet species, i.e. Tesla and SpaceX. Musk is a smart motherfucker, and he knows a ton about AI, and his sincere concern about this makes me scared." from: http://waitbutwhy.com/2015/05/elon-musk-the-worlds-raddest-man.html http://waitbutwhy.com/2015/05/elon-musk-the-worlds-raddest-m...
- xamdam 11y ago
  Yes, absolutely there is a reason for concern. I now volunteer for FLI (the organization Musk funded; we deal with both AI and other x-risks, synthetic bio etc). From a somewhat insider view I think there is a lot we can do, and time might be on our side (AGI is a pretty hard problem). I'm cautiously optimistic that we (rational and altruistic humans) can make a difference.
  - sixQuarks 11y ago
    keep up the good fight! Thank you for your efforts