3 ms·
If the first strong AI has goals that we don't like, and we aren't able to identify that and force it to change the goals by destroying it or reprogramming it -
by PeterisP 13y ago
If the first strong AI has goals that we don't like, and we aren't able to identify that and force it to change the goals by destroying it or reprogramming it - then "acting friendly" won't change these goals, and it will ensure that all future AI generations have it's initial buggy goals, not those that we might want.