This is not true of goal directed agents and all RLHF models are trained with, ahem, RL; see: "Optimal Policies Tend to Seek Power" from NeurIPS 2021. It's a v

This is not true of goal directed agents and all RLHF models are trained with, ahem, RL; see: "Optimal Policies Tend to Seek Power" from NeurIPS 2021. It's a very useful instrumental goal to be very powerful.