3 ms·
The authors referenced at the above link don't use that seemingly unwarranted tone of certainty. I can't see anything in Omohundro's papers of anything like the
by yesenadam 6y ago
The authors referenced at the above link don't use that seemingly unwarranted tone of certainty. I can't see anything in Omohundro's papers of anything like the same flavour, and Bostrom's conclusion is the sensible-sounding
It should be emphasized that the existence of convergent instrumental reasons, even if they apply to and are recognized by a particular agent, does not imply that the agent’s behavior is easily predictable. An agent might well think of ways of pursuing the relevant instrumental values that do not readily occur to us. This is especially true for a superintelligence, which could devise extremely clever but counterintuitive plans to realize its goals, possibly even exploiting as-yet undiscovered physical phenomena. What is predictable is that the convergent instrumental values would be pursued and used to realize the agent’s final goals, not the specific actions that the agent would take to achieve this.
- garmaine 6y agoI’m not what you’re picking up on other than academic prose. The very quote you give is very certain that convergent intermediate goals would be pursued.