4 ms·

I think that a lot of hallucination might just be due to not planning ahead - basically a case of running mouth before engaging brain, and then being in a situa

I think that a lot of hallucination might just be due to not planning ahead - basically a case of running mouth before engaging brain, and then being in a situation where one has uttered a bunch of nonsense - basically backed oneself into a conversational corner. A human might catch themselves with "err, never mind, forget that!", but the LLM's only recourse is to continue extrapolating the nonsense the only way it knows how - i.e. to bullshit/hallucinate the most statistically plausible continuation of the hole it dug for itself. As a simplistic made up example, say the training set included a bunch of statements about capital cities of various countries of the form "the capital of england is london", "the capital of france is paris", etc, but didn't include any data indicating the capital of australia... Now, if asked "what is the capital of australia?", it may confidently start "the capital of australia is ..." since this matches the pattern it learnt from the training set (with "australia" being copied from the context). However, when generating the next word the LLM (without realizing it) finds itself in the unfortunate situation of not having been trained on data that would let it predict the right answer, but it of course goes ahead and predicts as best it can anyway (i.e. hallucinates/bullshits) and maybe generates the name of some random important city in australia such as "sydney". The way to fix at least this cause (maybe the primary one) of hallucination is essentially to plan ahead .. don't start off saying "the capital of australia is .." without knowing where you are going with it! One approach might be additional training using tree of thought rollouts (generate multiple possible branching continuations for each training prompt) then evaluate them and use these as RL rewards to learn to predict words leading to good outcomes (e.g. "i don't know", rather than "the capital of australia is ..").

corethree 3y ago
As the query prompter you are in control of the feedback loop. You can ask the AI to re-examine it's output to catch errors just as a human would do for himself. Practically speaking this does work to a limited extent. Sometimes the AI just sticks with it's guns and runs with it just like a human might.
- HarHarVeryFunny 3y ago
  Sure, but I was addressing the issue of why "hallucinations" occur in the first place, and how to fix them. Having a human in the loop does seem pretty much required at the moment, but it doesn't help when asking the AI for the answer to something you don't know, and therefore not being able to realize that the confident answer was hallucinated and wrong. Of course multiple-answer quizzes are easier to get right by process of elimination, so the human might sometimes be able to say "THAT can't be right" and catch SOME of the hallucinations.
  - corethree 3y ago
    Oh you misunderstand. When you query the LLM for the second iteration of the loop just do it regardless. Say something generic, ask it to reanalyze the answer more carefully. Ask it to compare it with existing known data and check for logical consistency. You can do this EVEN if you don't know whether or not the answer is wrong. Because the input is generic you can make this automated. When a user makes a query create a feedback loop and feed that back into the neural network multiple times with additional input requests to re-analyze analyze the query and resulting output more carefully. You can do this until you exhaust all input nodes and it "forgets" what you talked about previously.
    - HarHarVeryFunny 3y ago
      Interesting - I'll have to try this!
- kromem 3y ago
  Tip: If you are adding a self-critique step, show the model the initial output as if it is evaluating something from someone else (i.e. "grade this answer from a student") as opposed to from itself (i.e. "you wrote this, is it really correct?"). As you correctly note, humans have a problem with admitting fault. Especially the case online. But humans online are very ready to correct others. That's exactly the kind of larger abstract pattern in the data a model would emulate.

corethree 3y ago

As the query prompter you are in control of the feedback loop. You can ask the AI to re-examine it's output to catch errors just as a human would do for himself. Practically speaking this does work to a limited extent. Sometimes the AI just sticks with it's guns and runs with it just like a human might.

HarHarVeryFunny 3y ago
Sure, but I was addressing the issue of why "hallucinations" occur in the first place, and how to fix them. Having a human in the loop does seem pretty much required at the moment, but it doesn't help when asking the AI for the answer to something you don't know, and therefore not being able to realize that the confident answer was hallucinated and wrong. Of course multiple-answer quizzes are easier to get right by process of elimination, so the human might sometimes be able to say "THAT can't be right" and catch SOME of the hallucinations.
- corethree 3y ago
  Oh you misunderstand. When you query the LLM for the second iteration of the loop just do it regardless. Say something generic, ask it to reanalyze the answer more carefully. Ask it to compare it with existing known data and check for logical consistency. You can do this EVEN if you don't know whether or not the answer is wrong. Because the input is generic you can make this automated. When a user makes a query create a feedback loop and feed that back into the neural network multiple times with additional input requests to re-analyze analyze the query and resulting output more carefully. You can do this until you exhaust all input nodes and it "forgets" what you talked about previously.
  - HarHarVeryFunny 3y ago
    Interesting - I'll have to try this!
kromem 3y ago
Tip: If you are adding a self-critique step, show the model the initial output as if it is evaluating something from someone else (i.e. "grade this answer from a student") as opposed to from itself (i.e. "you wrote this, is it really correct?"). As you correctly note, humans have a problem with admitting fault. Especially the case online. But humans online are very ready to correct others. That's exactly the kind of larger abstract pattern in the data a model would emulate.