> It's hard to balance the ability to create new, more useful utility functions with prevention of creating a utility function at odds with what the original en

> It's hard to balance the ability to create new, more useful utility functions with prevention of creating a utility function at odds with what the original entity valued. I think that for such an AI, the concept of a "more useful utility function" would be a nonsense. The AI's definition of 'useful' is the utility function. No other utility function can ever rate higher against the current utility function than the current utility function does.