11 ms·
Yes, I shouldn't anthropomorphize computers, they hate that. Anyway, yes, it will optimize whatever function it's told to optimize. So you'd better hope that w
by mikeash 11y ago
Yes, I shouldn't anthropomorphize computers, they hate that.
Anyway, yes, it will optimize whatever function it's told to optimize. So you'd better hope that whoever gives it that function 1) cares about people's wellbeing and 2) is good at expressing that in a way the computer can interpret.
1 is pretty obvious, 2 may be a bit less so. For example, a superintelligent computer commanded to maximize global average happiness might decide the best route would be to work towards killing the entire human population except for one individual, who is kept constantly drugged out of their mind.