3 ms·
More intelligent systems will find more efficient ways to achieve their goals than dumber ones (us). They will also select more effective intermediate goals and
by simonh 6y ago
More intelligent systems will find more efficient ways to achieve their goals than dumber ones (us). They will also select more effective intermediate goals and 'stepping stone' states towards their goals than we can anticipate. The consequence of this is that we cannot anticipate what means and intermediate objectives they will use. That's exactly what you are doing in your example, but attempting to speculate on that is futile.
That's what the paperclip maximiser example tries to show. In that example one of the intermediate goals to creating maximum paperclips is destroying human civilisation. It's not the ultimate objective, it's just an intermediate state necessary to efficient completion of the underlying goal. It's possible that in fact a superintelligent paperclip maximiser might work out a better strategy that uses human civilisation as a level to maximise paperclips, but maybe not. There's simply no way for us to be sure, because by definition it's smarter than us.
- LargoLasskhyfv 6y agoWe: "Send post cards, will ya?" They/it: "Sure thing, Kittenz!"