> Therefore the desire to get rid of the human race is actually a logical consequence of most utility functions, rather than being directly specified. That's o

> Therefore the desire to get rid of the human race is actually a logical consequence of most utility functions, rather than being directly specified. That's only true in the sense that there are infinitely many more Real numbers than whole numbers. Even so, I would argue there are infinitely many utility functions that directly involve the welfare of the human race, and you would be stupid to design an AI who didn't have a majority of its utility functions directly involving measures of human welfare. (Also, recognizing minimum welfare levels as well as median and average.)