4 ms·
> Far too often, people anthropomorphize AI or draw from fiction. Bostrom does his best to dispel these notions. He gets halfway to right. Recently there was a
by Zigurd 12y ago
> Far too often, people anthropomorphize AI or draw from fiction. Bostrom does his best to dispel these notions.
He gets halfway to right. Recently there was a thread about suicide on HN. Nearly all the comments on that thread ignored that humans, like other animals, are built to propagate the information in their genes, and argued against suicide based on concepts that are driven by or that rationalize subconscious drives. An AI, even one with a programmed goal of propagating itself, does not carry information in genes and the information in an AI was never selected for the actual ability to propagate in a larger system that thinks it is in control of its goals.
Real AIs are likely to be extremely alien. Even in warning people about anthropomorpizing, there's a lot of anthropomorphism. Evil AIs are depicted as human evil. So much of human motivation is closed to introspection that imagining alien and evil motivation is unlikely to be close to the eventual reality. This probably won't be widely understood until AIs are built. And what if they all choose to erase themsleves?