3 ms·
I've done a lot of thinking about this myself, and I think for an entity that thinks far enough ahead there would be. Lets say we have an AGI that is trying to
by psandersen 10y ago
I've done a lot of thinking about this myself, and I think for an entity that thinks far enough ahead there would be.
Lets say we have an AGI that is trying to maximise a goal, such as maximise its expected future knowledge over all time/space; an entity like this would likely 'play' extremely conservatively, slowly expand and build its understanding.
Now on a universal view the probability that it is the strongest is infinitesimal, there is always a chance of something bigger out there with a tech level that allows it to see without the AGI knowing about it. This is critical, as a min/max strategy would likely dictate playing extremely conservatively, and going out of your way to demonstrate that you're friendly, even when you think no one is watching and you're vastly superior. The universe is big, you'd have to be a real asshole to attack a sentient species when there is so much unused asteroid and other mass out there.
Need to formalise this more, as I think this approach holds the key to AI safety.
Tl;dr: On infinite time scales being the aggressor is too risky, therefore something at least approximating morality is optimal.