4 ms·

Really excited to see if this approach can help reduce hallucinations overall. Process supervision combined with the browsing models could drastically improve h

Really excited to see if this approach can help reduce hallucinations overall. Process supervision combined with the browsing models could drastically improve how logically sound anything that's generated is. Creating consistently valid logic is the harder part so this is awesome.

anonymousDan 3y ago
Is there some formal/objective definition of what exactly constitutes a 'hallucination' as opposed to other types of errors? At a high-level it only seems relevant with respect to questions for which there is some objectively true answer, or where 'facts' are included in answering a question that are false.
- golol 3y ago
  The paper states that hallucinations are logical reasoning errors... that confused me. For me a hallucination is the conjuring up of facts, things, references, characters etc. as is convenient for the given context.
  - BeetleB 3y ago
    Semantics aside: I once gave it a math problem, and its hallucinations really did present as logical errors. I was a TA for several years, and it was reminiscent of the mistakes students make. Something that sounds fairly plausible, but still incorrect, with logical errors or unjustified steps.
  - sebzim4500 3y ago
    It's really hard to formally define hallucination, for me it's a "know it when you see it" situation. In my view, saying "10 + 4 = 15" is not a hallucination but inventing a citation that doesn't exist is one. There is a big grey area between these, like what if it cites a real paper but gets the year wrong? Is that a hallucination (because the paper as cited is fake), or just an incorrect statement (because the year is just wrong)?
  - DiscourseFan 3y ago
    The trouble (probably not for basic math or even low-level analysis) is that what we refer to as "logic" or a logical reasoning error isn't uniform across different axiomatic systems, and the specific interpretations is a fairly human, social activity which cannot be readily "checked" in many cases for correctness against some baseline notion of "logic". The symbol φ, for instance, has a variety of significations in different context (the family of sets of all functions, probably something in physics idk), which our interpreto-bot (GPT) might not be able to both logically integrate and loosely interpret at the same time. Humans have the capacity to apprehend both consistent and complete logical systems: they interpret at the level of the text (at the level of the weave of signification), and any generally intelligent AI would have to mimic that behavior of the constant, on-the-fly dynamic changes to its network at the appearance of every new signifier in the same way as a human does.
- mbbbackus 3y ago
  Good point. You can create a formal definition of hallucinations in formal language, like math and logic, but probably not for natural language. There are bound to be edge cases where natural language defies the rules of formal logic without being incoherent. But, while you might not be able to have a simple formal definition, maybe you could create a model that's trained to recognize these edge cases, and the model would be a sort of approximation of a formal definition.
- User23 3y ago
  LLMs don’t have errors. What you perceive as an erroneous output is actually just one that failed to please you.
- akiselev 3y ago
  It’s a sliding scale that depends on the use case and different uses will want levels of hallucination that other uses would find unacceptable. In summarization, it’s literally anything that’s not in the context. If it’s used for writing historical fiction or scifi, very little would count as a hallucination as long as it’s following the prompt. In the recent case of the Texas lawyer, the citations follow a specific format and can be checked against a database, so hallucinations are easy to define w.r.t. citations.

anonymousDan 3y ago

Is there some formal/objective definition of what exactly constitutes a 'hallucination' as opposed to other types of errors? At a high-level it only seems relevant with respect to questions for which there is some objectively true answer, or where 'facts' are included in answering a question that are false.

golol 3y ago
The paper states that hallucinations are logical reasoning errors... that confused me. For me a hallucination is the conjuring up of facts, things, references, characters etc. as is convenient for the given context.
- BeetleB 3y ago
  Semantics aside: I once gave it a math problem, and its hallucinations really did present as logical errors. I was a TA for several years, and it was reminiscent of the mistakes students make. Something that sounds fairly plausible, but still incorrect, with logical errors or unjustified steps.
- sebzim4500 3y ago
  It's really hard to formally define hallucination, for me it's a "know it when you see it" situation. In my view, saying "10 + 4 = 15" is not a hallucination but inventing a citation that doesn't exist is one. There is a big grey area between these, like what if it cites a real paper but gets the year wrong? Is that a hallucination (because the paper as cited is fake), or just an incorrect statement (because the year is just wrong)?
- DiscourseFan 3y ago
  The trouble (probably not for basic math or even low-level analysis) is that what we refer to as "logic" or a logical reasoning error isn't uniform across different axiomatic systems, and the specific interpretations is a fairly human, social activity which cannot be readily "checked" in many cases for correctness against some baseline notion of "logic". The symbol φ, for instance, has a variety of significations in different context (the family of sets of all functions, probably something in physics idk), which our interpreto-bot (GPT) might not be able to both logically integrate and loosely interpret at the same time. Humans have the capacity to apprehend both consistent and complete logical systems: they interpret at the level of the text (at the level of the weave of signification), and any generally intelligent AI would have to mimic that behavior of the constant, on-the-fly dynamic changes to its network at the appearance of every new signifier in the same way as a human does.
mbbbackus 3y ago
Good point. You can create a formal definition of hallucinations in formal language, like math and logic, but probably not for natural language. There are bound to be edge cases where natural language defies the rules of formal logic without being incoherent. But, while you might not be able to have a simple formal definition, maybe you could create a model that's trained to recognize these edge cases, and the model would be a sort of approximation of a formal definition.
User23 3y ago
LLMs don’t have errors. What you perceive as an erroneous output is actually just one that failed to please you.
akiselev 3y ago
It’s a sliding scale that depends on the use case and different uses will want levels of hallucination that other uses would find unacceptable. In summarization, it’s literally anything that’s not in the context. If it’s used for writing historical fiction or scifi, very little would count as a hallucination as long as it’s following the prompt. In the recent case of the Texas lawyer, the citations follow a specific format and can be checked against a database, so hallucinations are easy to define w.r.t. citations.