3 ms·
I would like to note that the fear of something that could annihilate the human race is not an unreasonable one.
by megaduck 17y ago
I would like to note that the fear of something that could annihilate the human race is not an unreasonable one.
- mquander 17y agoWhether it's reasonable or unreasonable should not depend primarily on how frightening it is to you.
- megaduck 17y agoI'm sorry, I think the prospect of human extinction should be frightening to just about everybody. If we do eventually develop a trans-human AI, it's a virtual certainty that it will escape its "box". Whether it would kill us all is unknown. However, it definitely could, and we would be effectively powerless to stop it. "Reasonable" is a measure of risk tolerance. Since the downside risks associated with trans-human AIs are effectively infinite, fear of those risks is always reasonable. Corollary: Since the upside risks are also unbounded, greed is also reasonable. Frankly, though, I'd rather not roll those dice if we can avoid it.
- rbanffy 17y agoIf we can bring new intelligence to life, do we have the moral right to refrain from doing so? Also, would it be right to confine it to the box mentioned in the article while using its smarts to do useful work outside it? Shouldn't a trans-human AI be entitled a right to life?
- rbanffy 17y agoHomo sapiens will, eventually, be superseded by its descendants. A sufficiently intelligent super-human AI could annihilate us like we did the Neanderthals, but let's not forget we may be anthropomorphizing it a little bit too far here. It could also possibly be no more interested in us than we are to the yeast we use to make bread. Our survival depends on how annoying we are to them ;-)