3 ms·

I wonder if it's "fixed" or if it's just less obvious. It seems that an LLM would "hallucinate" a bogus answer if it didn't actually have a good answer somewher

by tjr 3y ago

I wonder if it's "fixed" or if it's just less obvious. It seems that an LLM would "hallucinate" a bogus answer if it didn't actually have a good answer somewhere in its training. Is GPT4 so much more trained that it rarely encounters something it doesn't have a reasonable answer for? In which case, it would still "hallucinate" if cornered on some more obscure matter? I mean, like, what would it mean to actually solve the problem? I would not expect any computer system to know the answer to literally everything, so the fix is not to train it more, but rather, for it to realize and acknowledge if it doesn't have enough data to give a good answer, and tell you that rather than make something up. Does GPT4 do that? (I have yet to use GPT4 myself.)

sebzim4500 3y ago
In my experience, GPT-4 is equally willing to make things up if it doesn't know something but it has so much more knowledge than GPT-3.5 that this happens less often in practice.
- mustacheemperor 3y ago
  On that note, I’ve found that just including in the prompt a request for GPT4 to consider its confidence level in an answer and inform me of that confidence level, to reconsider its answer if its confidence is low, and that accuracy is critically important for the topic of the conversation, also can result in better steering it. I mean, kind of works with humans too. In a high pressure work or school environment, people can confabulate the answers someone wants to hear to avoid discomfort “oh yes, we did the training exercise, the trucks and tanks are in great shape.” Sometimes people need to know they can admit they are wrong. I wonder if some aspect of current LLM tuning/training could be modified so LLMs are more “comfortable”, for lack of a non-anthropomorphized term coming to mind, with admitting they are unsure.
  - ukuina 3y ago
    When I tried this, it made up random, non-repeatable confidence values that had little bearing to the veracity of its statements.
    - sebzim4500 3y ago
      If you read the GPT-4 paper, they claim that the base model was well calibrated but the calibration was completely broken by RLHF

by tjr 3y ago

sebzim4500 3y ago

In my experience, GPT-4 is equally willing to make things up if it doesn't know something but it has so much more knowledge than GPT-3.5 that this happens less often in practice.

mustacheemperor 3y ago
On that note, I’ve found that just including in the prompt a request for GPT4 to consider its confidence level in an answer and inform me of that confidence level, to reconsider its answer if its confidence is low, and that accuracy is critically important for the topic of the conversation, also can result in better steering it. I mean, kind of works with humans too. In a high pressure work or school environment, people can confabulate the answers someone wants to hear to avoid discomfort “oh yes, we did the training exercise, the trucks and tanks are in great shape.” Sometimes people need to know they can admit they are wrong. I wonder if some aspect of current LLM tuning/training could be modified so LLMs are more “comfortable”, for lack of a non-anthropomorphized term coming to mind, with admitting they are unsure.
- ukuina 3y ago
  When I tried this, it made up random, non-repeatable confidence values that had little bearing to the veracity of its statements.
  - sebzim4500 3y ago
    If you read the GPT-4 paper, they claim that the base model was well calibrated but the calibration was completely broken by RLHF