This seems related to the idea of coherent extrapolated volition (https://intelligence.org/files/CEV.pdf https://intelligence.org/files/CEV.pdf). Both have some

This seems related to the idea of coherent extrapolated volition (https://intelligence.org/files/CEV.pdf https://intelligence.org/files/CEV.pdf). Both have some of the same problems--in particular, setting up the system requires making moral judgments about how to do so, so it's not actually value-neutral. (Aside: If I have two completely different thoughts about an article, should I post them in two separate comments or in the same comment?)