3 ms·

There is a clear hypothesis: that a sufficiently advanced AI will manipulate most humans toward its own goals, and even its creators won't be able to stop it on

by tlb 15y ago

There is a clear hypothesis: that a sufficiently advanced AI will manipulate most humans toward its own goals, and even its creators won't be able to stop it once it gets going. So the stakes really are "saving the world". This hypothesis isn't easy to test. That doesn't make it unscientific -- cosmology and evolutionary theories are also hard to test.

jonmc12 15y ago
Thats not a hypothesis in the scientific sense, that is a scientifically-based speculation. http://en.wikipedia.org/wiki/Scientific_method http://en.wikipedia.org/wiki/Scientific_method btw, does SIAI, et al, actually align to a common speculation that logically unfolds to the notion of 'saving the world'? Could you send me a link? The best I could find is in this pdf - http://singinst.org/upload/artificial-intelligence-risk.pdf http://singinst.org/upload/artificial-intelligence-risk.pdf - pages 18, 19 gives an overview of this paper: http://singinst.org/upload/artificial-intelligence-risk.pdf http://singinst.org/upload/artificial-intelligence-risk.pdf I believe the paper is intended to explore possible risks - not to state scientific hypotheses. Is it science that a super-intelligence will emerge at all? Is it science what the timing of the onset of super-intelligence will be? How about the transition speed of the onset? Is it proven or accepted that this notion of 'manipulation of human goals' is meaningfully defined or even a negative thing? Is it proven that Friendly AI would prevent any set of bad scenarios from occurring? No.. of course not. Each of these points is discussed as speculation in a general discussion of AI risks. To take a stacked set of speculations, and say you are 'saving the world' - I don't know how you get there without religion.
- endtime 15y ago
  I don't think they really claim that. I think that they'd rather say "a singularity is plausible and its effect would likely be of very high magnitude, so even marginally positively increasing the kind of singularity that might come about has expected value on the level of saving the world."

by tlb 15y ago

jonmc12 15y ago

Thats not a hypothesis in the scientific sense, that is a scientifically-based speculation. http://en.wikipedia.org/wiki/Scientific_method http://en.wikipedia.org/wiki/Scientific_method btw, does SIAI, et al, actually align to a common speculation that logically unfolds to the notion of 'saving the world'? Could you send me a link? The best I could find is in this pdf - http://singinst.org/upload/artificial-intelligence-risk.pdf http://singinst.org/upload/artificial-intelligence-risk.pdf - pages 18, 19 gives an overview of this paper: http://singinst.org/upload/artificial-intelligence-risk.pdf http://singinst.org/upload/artificial-intelligence-risk.pdf I believe the paper is intended to explore possible risks - not to state scientific hypotheses. Is it science that a super-intelligence will emerge at all? Is it science what the timing of the onset of super-intelligence will be? How about the transition speed of the onset? Is it proven or accepted that this notion of 'manipulation of human goals' is meaningfully defined or even a negative thing? Is it proven that Friendly AI would prevent any set of bad scenarios from occurring? No.. of course not. Each of these points is discussed as speculation in a general discussion of AI risks. To take a stacked set of speculations, and say you are 'saving the world' - I don't know how you get there without religion.

endtime 15y ago
I don't think they really claim that. I think that they'd rather say "a singularity is plausible and its effect would likely be of very high magnitude, so even marginally positively increasing the kind of singularity that might come about has expected value on the level of saving the world."