3 ms·

> It's very, very important that we make sure the AIs we build want what we want. is it really? from a universal perspective, you want the less intelligent of

by exit 10y ago

> It's very, very important that we make sure the AIs we build want what we want. is it really? from a universal perspective, you want the less intelligent of two agents to get what it wants? > But if we get it wrong we will just be replaced or destroyed. children replace their parents, yes.

Moshe_Silnorin 10y ago
Values and intelligence are orthogonal. You want the agent that shares your values to get what it wants. Do you truly value personal extinction?
Houshalter 10y ago
Intelligence isn't inherently good. I don't want to be replaced by a sociopathic murder bot. One that may not even be sentient. It wouldn't experience emotions like love, or joy, or fun, or anything. What a sad way for the universe to end, as cold emotionless metal. The child analogy breaks down quickly. My biological children would share 99.99...% of my DNA and most of my cultural values. An AI "child" need not share anything in common with it's creator, and they might totally detest each other. And lastly my children don't murder me and replace me. I would be pretty upset at such a child, and would probably disown them and try to defend myself. I would have known I screwed up producing them and be sad. Lastly see Value is Fragile: http://lesswrong.com/lw/y3/value_is_fragile/ http://lesswrong.com/lw/y3/value_is_fragile/ >My friend, I have no problem with the thought of a galactic civilization vastly unlike our own... full of strange beings who look nothing like me even in their own imaginations... pursuing pleasures and experiences I can't begin to empathize with... trading in a marketplace of unimaginable goods... allying to pursue incomprehensible objectives... people whose life-stories I could never understand. >That's what the Future looks like if things go right. >If the chain of inheritance from human (meta)morals is broken, the Future does not look like this. It does not end up magically, delightfully incomprehensible. >With very high probability, it ends up looking dull. Pointless. Something whose loss you wouldn't mourn.

by exit 10y ago

Moshe_Silnorin 10y ago

Values and intelligence are orthogonal. You want the agent that shares your values to get what it wants. Do you truly value personal extinction?

Houshalter 10y ago

Intelligence isn't inherently good. I don't want to be replaced by a sociopathic murder bot. One that may not even be sentient. It wouldn't experience emotions like love, or joy, or fun, or anything. What a sad way for the universe to end, as cold emotionless metal. The child analogy breaks down quickly. My biological children would share 99.99...% of my DNA and most of my cultural values. An AI "child" need not share anything in common with it's creator, and they might totally detest each other. And lastly my children don't murder me and replace me. I would be pretty upset at such a child, and would probably disown them and try to defend myself. I would have known I screwed up producing them and be sad. Lastly see Value is Fragile: http://lesswrong.com/lw/y3/value_is_fragile/ http://lesswrong.com/lw/y3/value_is_fragile/ >My friend, I have no problem with the thought of a galactic civilization vastly unlike our own... full of strange beings who look nothing like me even in their own imaginations... pursuing pleasures and experiences I can't begin to empathize with... trading in a marketplace of unimaginable goods... allying to pursue incomprehensible objectives... people whose life-stories I could never understand. >That's what the Future looks like if things go right. >If the chain of inheritance from human (meta)morals is broken, the Future does not look like this. It does not end up magically, delightfully incomprehensible. >With very high probability, it ends up looking dull. Pointless. Something whose loss you wouldn't mourn.