It's not just cute, it's an open problem in the theory of recursively self-modifying AI design--there's no general solution to an AI hijacking its own reward ch

It's not just cute, it's an open problem in the theory of recursively self-modifying AI design--there's no general solution to an AI hijacking its own reward channel, so far. https://www.lesswrong.com/posts/upLot6eG8cbXdKiFS/reward-function-learning-the-learning-process https://www.lesswrong.com/posts/upLot6eG8cbXdKiFS/reward-fun... is the most recent thing I know of in the area.