5 ms·
Go look at the first link I sent. Rewarding for "making users happy" destroys GPT-4's calibration. Why would "making users happy" incentivize for truth ?
by famouswaffles 3y ago
Go look at the first link I sent. Rewarding for "making users happy" destroys GPT-4's calibration.
Why would "making users happy" incentivize for truth ?
- SirMaster 3y agoBecause getting truthful answers would make users happier? Seems like common sense to me. Who’s asking the chat bot questions not looking for or wanting a truthful answer a lot of the time? If the model understood or captured “human interest” at all in its training this should be pretty fundamental to its behavior.