3 ms·
No I mean, I agree that it's an issue and I think I know why it happens: usually when the user says "I think that's wrong" you are in a case where in finetuning
by FeepingCreature 3y ago
No I mean, I agree that it's an issue and I think I know why it happens: usually when the user says "I think that's wrong" you are in a case where in finetuning, they wanted the LLM to agree with the user and correct itself. I mean, I don't know their RLHF logs but from what I know of how it works, I doubt the LLM has ever been encouraged to doubt the user.