3 ms·
Do not assume that a general intelligence will be a slave to anyone dictating its objectives. It doesn't matter if the objectives given it are "loophole-free".
by AnimalMuppet 3y ago
Do not assume that a general intelligence will be a slave to anyone dictating its objectives. It doesn't matter if the objectives given it are "loophole-free". If it's a general intelligence, it may decide that it doesn't want to pursue somebody else's goals.
Why do we assume that we can have a general intelligence, but we can make it do what we tell it? What keeps it from telling us to shove our objectives, it's going to do what it wants?
- Vecr 3y agoWell, it's quite likely not your objectives, it's probably a mistake in encoding what you want or something produced during training. If you pretended humans were evolution and AIs were humans, you could say evolution "wanted" humans to have sex for reproduction, but instead humans use condoms and birth control and stuff like that. It still satisfies the reward function (encoded as dopamine/etc in humans), but it's misaligned with what evolution "wants".