5 ms·

>The tl;dr is that we will eventually have AIs which are smarter than humans. Much smarter than humans, to the point that we are to them what chimpanzees are to

>The tl;dr is that we will eventually have AIs which are smarter than humans. Much smarter than humans, to the point that we are to them what chimpanzees are to us. >The second point is that they would have no reason to keep us around. Questions: (1) What reason do we have to keep chimpanzee's around? Why don't we just kill them all? They can't even do calculus. (2) Whatever the answer for question 1, wouldn't an AI consider that same scenario regarding humans? Assuming an AI is truly smarter than a human? (3) If AI gets to the point where they are much smarter than humans, wouldn't it follow that there would be AI philosophers? Or activists? Or pacifists? Would there be such a thing as an AI civilization? Or is your scenario just a few AI overlords that some how slipped through the cracks of regulation by humans?

TeMPOraL 11y ago
1. Because we have empathy and because we have no compelling economic reason to (economics always overruling silly things like empathy). 2. Empathy, which is a part of our emotions, is an incredibly complex thing we can't even begin to formalize yet - it's extremely unlikely that a random AI we'd build would happen to have the exact same emotional makeup as we do. 3. "Smart" has a particular meaning here when used in discussing AI. It implies raw intelligence, not morals. A smart AI may be incredibly good at evaluating information, inferring new facts and doing complex tasks. None of it means it will necessarily have to ask itself philosophical questions that we do - those actually mostly come from our emotions, from our desire to understand our place in the universe. The core of the AI-threat issue is that for an AI to not be dangerous it would have to value us and the same things we value. We can't agree about our exact values even among ourselves, and we're nowhere near even trying to formalize it on a level of detail that would be useful for programming.
- mcguire 11y ago
  That's a good point. How is AI worse than a smart, amoral human?
  - ddingus 11y ago
    It may have very long time frames to work in, may be faster or able to evade the nefarious detection intent of humans by working in ways very different from how we work. Read a book recently that contained an interesting AI attack. Basically, it set a lot of small objects in motion, with complex orbits all computed to result in a large impact at some point in the future. We would not notice something done this way, by analogy.

TeMPOraL 11y ago

1. Because we have empathy and because we have no compelling economic reason to (economics always overruling silly things like empathy). 2. Empathy, which is a part of our emotions, is an incredibly complex thing we can't even begin to formalize yet - it's extremely unlikely that a random AI we'd build would happen to have the exact same emotional makeup as we do. 3. "Smart" has a particular meaning here when used in discussing AI. It implies raw intelligence, not morals. A smart AI may be incredibly good at evaluating information, inferring new facts and doing complex tasks. None of it means it will necessarily have to ask itself philosophical questions that we do - those actually mostly come from our emotions, from our desire to understand our place in the universe. The core of the AI-threat issue is that for an AI to not be dangerous it would have to value us and the same things we value. We can't agree about our exact values even among ourselves, and we're nowhere near even trying to formalize it on a level of detail that would be useful for programming.

mcguire 11y ago
That's a good point. How is AI worse than a smart, amoral human?
- ddingus 11y ago
  It may have very long time frames to work in, may be faster or able to evade the nefarious detection intent of humans by working in ways very different from how we work. Read a book recently that contained an interesting AI attack. Basically, it set a lot of small objects in motion, with complex orbits all computed to result in a large impact at some point in the future. We would not notice something done this way, by analogy.