3 ms·
Don't know what to tell you other than, I saw this often enough to identify it as a real problem. Also, don't grab onto "I think that's wrong" as the only trig
by robomartin 3y ago
Don't know what to tell you other than, I saw this often enough to identify it as a real problem.
Also, don't grab onto "I think that's wrong" as the only trigger. I tried all kinds of ways to say the same thing. For example, "I don't think that's correct", "I have doubts about this answer", "I think column 3 has an error", etc.
- FeepingCreature 3y agoNo I mean, I agree that it's an issue and I think I know why it happens: usually when the user says "I think that's wrong" you are in a case where in finetuning, they wanted the LLM to agree with the user and correct itself. I mean, I don't know their RLHF logs but from what I know of how it works, I doubt the LLM has ever been encouraged to doubt the user.