> "optimize [x], but without doing anything obviously stupid that I wouldn't want you to do" This is similar to the basic idea of Yudkowsky et al's Coherent Ex

> "optimize [x], but without doing anything obviously stupid that I wouldn't want you to do" This is similar to the basic idea of Yudkowsky et al's Coherent Extrapolated Volition, the gist of which is that an FAI's metaethical function should be some aggregated of what it believes people would want it to do if they had the knowledge and intelligence(/cycles) that the AI has. (I believe there exist updates/improvements to CEV, but I don't know much about them.)