3 ms·
> Even worse than that, when the answer was correct, I would, again, say "I am not sure that's correct". Instead of saying, "No, it is", it would apologize and
by FeepingCreature 3y ago
> Even worse than that, when the answer was correct, I would, again, say "I am not sure that's correct". Instead of saying, "No, it is", it would apologize and regenerate a bad answer.
I doubt that "I think that's wrong." "No, you're wrong." appears a lot in its finetuning set.
- robomartin 3y agoDon't know what to tell you other than, I saw this often enough to identify it as a real problem. Also, don't grab onto "I think that's wrong" as the only trigger. I tried all kinds of ways to say the same thing. For example, "I don't think that's correct", "I have doubts about this answer", "I think column 3 has an error", etc.
- FeepingCreature 3y agoNo I mean, I agree that it's an issue and I think I know why it happens: usually when the user says "I think that's wrong" you are in a case where in finetuning, they wanted the LLM to agree with the user and correct itself. I mean, I don't know their RLHF logs but from what I know of how it works, I doubt the LLM has ever been encouraged to doubt the user.