3 ms·
Interesting. I wonder if its due to it having been trained on data from Humans. E.g. I guess humans “scheme” often when for example selling things, like in adve
by koe123 2y ago
Interesting. I wonder if its due to it having been trained on data from Humans. E.g. I guess humans “scheme” often when for example selling things, like in advertising: “I know how to solve your problem, and the best way is with my thing!”.
In that view perhaps the contrary, the thing not scheming, would be more surprising.
- theptip 2y agoThe prediction from the LessWrong folks was that it’s inevitable that a rational actor with “goals” would do this, so models trained purely on RL would exhibit this too. (Instrumental Convergence is the name for the theory that predicts power-seeking in a generalized way.) I agree that we should expect LLMs to be particularly vulnerable to this as you note. But it seems to me that LLMs seem to be absorbing some understanding of human morality too, which might make it possible to steer them into “the best of us” territory.