4 ms·
I've also noticed LLMs seem to lack conviction on the correctness of their answers. As the paper notes, you can easily convince the transformer that a correct a
by pton_xd 3y ago
I've also noticed LLMs seem to lack conviction on the correctness of their answers. As the paper notes, you can easily convince the transformer that a correct answer is wrong, and needs adjustment. Ultimately they're just trying to please you. For example with ChatGPT 3.5 (abbreviated):
me: what is sin -pi/2
gpt: -1
me: that's not right
gpt: I apologize, let me clarify, the answer is 1
- hellcow 3y agoI just re-ran this on GPT-4 and it apologized, told me I was right, and then said again that the answer was -1. So while it lacked conviction it at least kept the correct answer.
- muzani 3y agogpt-4: Actually, the value of \(\sin(-\pi/2)\) is indeed \(-1\). The sine function represents the y-coordinate of a point on the unit circle corresponding to a given angle. At \(-\pi/2\) radians, which is equivalent to 270 degrees or a quarter circle in the negative direction, the point on the unit circle is at the bottom with coordinates (0, -1). Therefore, the sine of \(-\pi/2\) is \(-1\). ===== The smarter it is, the more conviction it has. GPT-3.5 has a lot of impostor syndrome and it's probably deserved lol. But GPT-4 starts to stutter when you give it enough math questions, which aren't its forte.
- muzani 3y agoIf anything, GPT-4 has the opposite problem. Ask it to check your homework and it'll go "20/5 is not 4. The correct answer is 4"
- kaiokendev 3y agoThis is due to the RLHF alignment, only product-focused. It would be very annoying for users to fight back and forth with the LLM on the correctness of the answer, especially when it is so prone to hallucination.