3 ms·
It's at least sometimes a "Clever Hans" effect, or an "ELIZA" effect. These models have been RLHFed to just go along and agree with whatever the user asks for,
by roywiggins 2y ago
It's at least sometimes a "Clever Hans" effect, or an "ELIZA" effect.
These models have been RLHFed to just go along and agree with whatever the user asks for, even if the user doesn't realize what's going on. This is a bit of a cheap trick- of course users will find something more intelligent if it agrees with them. When models start arguing back you get stuff like early Bing Chat going totally deranged.