3 ms·

The secondary psychiatric meaning is much more precisely what LLMs do than "hallucination", though, which counts for something.

KaiserPro 3y ago
It really doesn't the secondary meaning describes a symptom of brain disorder. > The hypothesis is that the patient generates information as a compensatory mechanism to fill holes in one’s memories. It functions for self-coherence, integration of memories, and self-relevance. [1] Which is not what the LLM is doing. The LLM is choosing a statistically relevant next token, which is wrong because its missed some context. There are much better psych terms out there, hell, even aphasia is a better term, but its still wrong. What it actually is, is noise. Its noise that dilutes the signal we really want. but thats much harder to use, and you won't be remembered as the person who coined that phrase. [1]https://www.ncbi.nlm.nih.gov/books/NBK536961/#:~:text=Confabulation%20is%20a%20neuropsychiatric%20disorder,without%20the%20intention%20of%20deceit https://www.ncbi.nlm.nih.gov/books/NBK536961/#:~:text=Confab....
- dimal 3y ago
  You make two statements that you seem to think that are at odds with each other, but I don’t think that’s the case. > The hypothesis is that the patient generates information as a compensatory mechanism to fill holes in one’s memories. It functions for self-coherence, integration of memories, and self-relevance. > The LLM is choosing a statistically relevant next token, which is wrong because its missed some context. The reason, both cases, is that there is a lack of context. The reason why the next statistically probable token is chosen is because that is the one that provides coherence. It seems like a similar process.
- randallsquared 3y ago
  The term “confabulation” is a label for observed actions. You’ve pivoted to a hypothesis about the cause of the behavior in humans, which is likely, to put it mildly, to be different in detail from the cause of the behavior in current-day LLMs, but the behavior itself is still similar enough to share a moniker, in my opinion.
  - KaiserPro 3y ago
    > You’ve pivoted to a hypothesis about the cause of the behavior in humans, Its a psychology, the hypothesis is critical to the diagnosis, thats why several conditions are contested. It requires a lot of subjective analysis to diagnose. For example one difference between false memory and confabulation is dependent on the observer believing that the subject was given a trigger or not. False memories also depend on social cues. similarly the difference between schizophrenia and psychosis is pretty thin, and can depend on whether the person diagnosing you believes there was a trigger, or its "genetic"/long term [1]. Psych is really not a well regulated science, and shouldn't be used to describe anything LLM based. As I've asserted before, its better described as noise, rather than an anthropomorphic label based on suspicious science. [1] as always its a bit more complex. but only a touch.

KaiserPro 3y ago

It really doesn't the secondary meaning describes a symptom of brain disorder. > The hypothesis is that the patient generates information as a compensatory mechanism to fill holes in one’s memories. It functions for self-coherence, integration of memories, and self-relevance. [1] Which is not what the LLM is doing. The LLM is choosing a statistically relevant next token, which is wrong because its missed some context. There are much better psych terms out there, hell, even aphasia is a better term, but its still wrong. What it actually is, is noise. Its noise that dilutes the signal we really want. but thats much harder to use, and you won't be remembered as the person who coined that phrase. [1]https://www.ncbi.nlm.nih.gov/books/NBK536961/#:~:text=Confabulation%20is%20a%20neuropsychiatric%20disorder,without%20the%20intention%20of%20deceit https://www.ncbi.nlm.nih.gov/books/NBK536961/#:~:text=Confab....

dimal 3y ago
You make two statements that you seem to think that are at odds with each other, but I don’t think that’s the case. > The hypothesis is that the patient generates information as a compensatory mechanism to fill holes in one’s memories. It functions for self-coherence, integration of memories, and self-relevance. > The LLM is choosing a statistically relevant next token, which is wrong because its missed some context. The reason, both cases, is that there is a lack of context. The reason why the next statistically probable token is chosen is because that is the one that provides coherence. It seems like a similar process.
randallsquared 3y ago
The term “confabulation” is a label for observed actions. You’ve pivoted to a hypothesis about the cause of the behavior in humans, which is likely, to put it mildly, to be different in detail from the cause of the behavior in current-day LLMs, but the behavior itself is still similar enough to share a moniker, in my opinion.
- KaiserPro 3y ago
  > You’ve pivoted to a hypothesis about the cause of the behavior in humans, Its a psychology, the hypothesis is critical to the diagnosis, thats why several conditions are contested. It requires a lot of subjective analysis to diagnose. For example one difference between false memory and confabulation is dependent on the observer believing that the subject was given a trigger or not. False memories also depend on social cues. similarly the difference between schizophrenia and psychosis is pretty thin, and can depend on whether the person diagnosing you believes there was a trigger, or its "genetic"/long term [1]. Psych is really not a well regulated science, and shouldn't be used to describe anything LLM based. As I've asserted before, its better described as noise, rather than an anthropomorphic label based on suspicious science. [1] as always its a bit more complex. but only a touch.