> While interesting, this article seems to ignore the possibility that most people simply have a very strong desire to live. Alternatively: People can have goa

> While interesting, this article seems to ignore the possibility that most people simply have a very strong desire to live. Alternatively: People can have goals related to the actual outside state of the world, rather than their own internal state. If you can only conceptualize goals in terms of your own internal state, then sure, you might kill yourself to avoid pain, or wirehead yourself at the first opportunity. But if your actual goals are out there in the world, then it's essentially as you say above. Dying reduces your influence on the world. If you're not alive, it's much harder to steer the world to your desired state. Survival is a convergent instrumental goal, for people as much as for AIs.