3 ms·
The Three Body Problem and ASI are two songs on the same instrument: value misalignment in cases of massive power differentials. In an indifferent universe, tw
by kerbalspacepro 8y ago
The Three Body Problem and ASI are two songs on the same instrument: value misalignment in cases of massive power differentials.
In an indifferent universe, two optimizers optimizing different things will be in conflict. Can we build machines that optimize what we want to optimize? Can we align our optimization with aliens before it becomes optimal for one of us to annihilate the other?