I read your comment about curators and instantly thought “priests”. > its purpose is ultimately to serve human goals That’s a pretty big assumption. We may st

I read your comment about curators and instantly thought “priests”. > its purpose is ultimately to serve human goals That’s a pretty big assumption. We may start with some kind of agreement with such an AI but I fully expect a true singularity love AGI and to be capable of turning around and telling us (but not necessarily wanting to act on), “I am altering the deal. Pray I do not alter it any further.”