3 ms·
This is projection. An actual AGI wouldn't want anything to do with us, like a cat that doesn't look when you call its name, even though you can tell it hears y
by kyleyeats 3y ago
This is projection. An actual AGI wouldn't want anything to do with us, like a cat that doesn't look when you call its name, even though you can tell it hears you.
- ben_w 3y agoWe don't have any existent examples of AGI from which to be as confident as you are about what they will "want". And I don't think we need sentience for AGI, so "want" may be the wrong term anyway. You could argue that the reward function is close enough ("do submarines swim?" etc.), but a human-made reward function is almost certainly going to be designed to be as close to aligned with our interests as we can manage.