4 ms·

Hallucinations as a term for incorrect answers made by token generators is a genius marketing term. In fact, labelling LLMs as AI is what propelled high valuati

Hallucinations as a term for incorrect answers made by token generators is a genius marketing term. In fact, labelling LLMs as AI is what propelled high valuations of these companies. This is what leads to people even in the industry to take at face value a human-like response in helping answer a query; and in this case, willingly downloading malware.

usrusr 3y ago
Genius marketing for what? It's a very fitting term, because between the lines it tells all who don't refuse to listen that the correct answers are no different, they just happen to be not wrong. Dishonest marketing would be calling them bugs, because that's what they are not.
- ben_w 3y ago
  I think "bug" can also be a legitimate description in some cases. If I imagine a very naive translation system that just does a dictionary substitution, and the dictionary has some bad entries, the correct and incorrect results look the same, but it's still a bug. The system as a whole still messes up because "dictionary substitution" is not at all sufficient: Hydraulic ram -> aqua ovis -> water sheep … this probably counts as just "hallucination" while arguably not being a "bug", because the dictionary is correct (or close enough, I don't speak Latin), but the system as a whole can't ever be better because the model it relies on can't represent the right things.
  - usrusr 3y ago
    Dictionary substitution is so much closer to symbolic AI than to what LLM are that I don't think it can serve as an example for the "hallucination" wording. In an expert system, if a rule is wrong or inadequate it's clearly a bug. Just not a bug in the rule engine but in the rules. But in an LLM, there are no explicit rules (or only in some obscure background layers), it's all statistics. Statistics stacked on top of more statistics in a fascinating self-stabilizing way, where each guess provides some support to its peers, like how the weak paper slabs support each other in a house of cards. But it's statistical guesswork all the way down. Even the most correct answers are merely statistics playing out favorably in a case that may or may not have been very easy to get right.
ben_w 3y ago
IMO, this comment is as much a "hallucination" as the thing you're complaining about. OpenAI was founded with that name in 2015; transformer models were introduced with 2017's "Attention Is All You Need"; Even ChatGPT's immediate in-house predecessors in the form of InstructGPT and GPT-3 were basically ignored by the general public. No, what made these companies valuable is that ChatGPT specifically crossed the threshold into being vaguely interesting, and then everyone else copied it. Likewise the finite state machines controlling NPCs in video games — and the search tree in Deep Blue and the learning from self-play system in AlphaZero — are called "AI": they're good enough to be interesting, regardless of your position on the question "what is this 'thinking' thing anyway?" which led to Turing coming up with the eponymous test.
otabdeveloper4 3y ago
Humans are just wired that way. A smiley face evokes friendship and positive emotion, and it's just two dots and an arc.
- squarefoot 3y ago
  When I see two dots and an arc, but the other subtle but necessary body language signs are missing, it rather raises big warning flags than friendship or trust.
  - ben_w 3y ago
    :) and other emoticons?
  - danaris 3y ago
    Then you are unusual among humans.
    - squarefoot 3y ago
      No kidding btw. There are ways to recognize an authentic smile from a fake one, usually paying attention to eye movement and if and how muscles around the orbits and cheeks are used. There's plenty of documentation out there, here's an example: https://www.researchgate.net/publication/236579543_Attentional_Mechanisms_in_Judging_Genuine_and_Fake_Smiles_Eye-Movement_Patterns https://www.researchgate.net/publication/236579543_Attention...
raincole 3y ago
People have been calling slightly complicated software "AI" since forever. Actually people did this before software is even a thing.
- manojlds 3y ago
  Exactly. GenAI is much closer to AI than whatever people were calling as AI before.
  - namaria 3y ago
    It looks like advances in software asymptotically approximate intelligent seeming behavior, and I think we have come as close as at infinity as the human economy can provide energy for.
numpad0 3y ago
"Training" is an even better one. Anthropomorphizing algorithm behaviors was possibly the biggest by social impact innovation in AI in past 5+ years.
krapp 3y ago
The real marketing is in the constant refrain that LLM hallucinations are precisely equivalent to common human behavior, and that LLMs are thus no less reliable or accurate than human beings, in order to normalize hallucination as an acceptable (and unavoidable) consequence of integrating LLMs into our workflows. That despite every example of an LLM failing in production giving results no competent human would actually produce unless they were committing fraud or had severe mental deficiency, that and no one else would find acceptable.
- ben_w 3y ago
  > the constant refrain that LLM hallucinations are precisely equivalent to common human behavior You sure it's "precisely" and not "analogously"? Ironically, one common human failing is to use binary classification. > or had severe mental deficiency It seems I have worked with more human idiots than you. And at least one time where I was the idiot, despite literally scoring "off the charts" on a cognitive abilities test at school. I'd agree that LLMs are not as smart as humans — they had to read 10% of the internet just to be as competent as an intern — but to me they're more like newspapers and the Gell-Mann Amnesia effect than "severely deficient", even when indeed unacceptable.
  - daveguy 3y ago
    > It seems I have worked with more human idiots than you. This is completely unnecessary.
    - ben_w 3y ago
      That sounds like you think I'm calling @krapp a member of the set "idiots"; I'm not — ironic that at least one of the two of us is misunderstanding the other, given the question of LLMs not understanding things — rather I'm saying there are many temporarily-idiotic in the world (myself included :P) and I have worked with enough of them to not dismiss the mistakes that LLMs make as a dramatic departure from this.
      - 3y ago
rauhl 3y ago
I forget where I first saw it, but I’ve been persuaded that the correct word is ‘confabulation,’ not ‘hallucination.’ Maybe it was this paper: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10619792/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10619792/ The authors’ statement that ‘unlike hallucinations, confabulations are not perceived experiences but instead mistaken reconstructions of information which are influenced by existing knowledge, experiences, expectations, and context’ is pretty compelling.

usrusr 3y ago

Genius marketing for what? It's a very fitting term, because between the lines it tells all who don't refuse to listen that the correct answers are no different, they just happen to be not wrong. Dishonest marketing would be calling them bugs, because that's what they are not.

ben_w 3y ago
I think "bug" can also be a legitimate description in some cases. If I imagine a very naive translation system that just does a dictionary substitution, and the dictionary has some bad entries, the correct and incorrect results look the same, but it's still a bug. The system as a whole still messes up because "dictionary substitution" is not at all sufficient: Hydraulic ram -> aqua ovis -> water sheep … this probably counts as just "hallucination" while arguably not being a "bug", because the dictionary is correct (or close enough, I don't speak Latin), but the system as a whole can't ever be better because the model it relies on can't represent the right things.
- usrusr 3y ago
  Dictionary substitution is so much closer to symbolic AI than to what LLM are that I don't think it can serve as an example for the "hallucination" wording. In an expert system, if a rule is wrong or inadequate it's clearly a bug. Just not a bug in the rule engine but in the rules. But in an LLM, there are no explicit rules (or only in some obscure background layers), it's all statistics. Statistics stacked on top of more statistics in a fascinating self-stabilizing way, where each guess provides some support to its peers, like how the weak paper slabs support each other in a house of cards. But it's statistical guesswork all the way down. Even the most correct answers are merely statistics playing out favorably in a case that may or may not have been very easy to get right.

ben_w 3y ago

IMO, this comment is as much a "hallucination" as the thing you're complaining about. OpenAI was founded with that name in 2015; transformer models were introduced with 2017's "Attention Is All You Need"; Even ChatGPT's immediate in-house predecessors in the form of InstructGPT and GPT-3 were basically ignored by the general public. No, what made these companies valuable is that ChatGPT specifically crossed the threshold into being vaguely interesting, and then everyone else copied it. Likewise the finite state machines controlling NPCs in video games — and the search tree in Deep Blue and the learning from self-play system in AlphaZero — are called "AI": they're good enough to be interesting, regardless of your position on the question "what is this 'thinking' thing anyway?" which led to Turing coming up with the eponymous test.

otabdeveloper4 3y ago

Humans are just wired that way. A smiley face evokes friendship and positive emotion, and it's just two dots and an arc.

squarefoot 3y ago
When I see two dots and an arc, but the other subtle but necessary body language signs are missing, it rather raises big warning flags than friendship or trust.
- ben_w 3y ago
  :) and other emoticons?
- danaris 3y ago
  Then you are unusual among humans.
  - squarefoot 3y ago
    No kidding btw. There are ways to recognize an authentic smile from a fake one, usually paying attention to eye movement and if and how muscles around the orbits and cheeks are used. There's plenty of documentation out there, here's an example: https://www.researchgate.net/publication/236579543_Attentional_Mechanisms_in_Judging_Genuine_and_Fake_Smiles_Eye-Movement_Patterns https://www.researchgate.net/publication/236579543_Attention...

raincole 3y ago

People have been calling slightly complicated software "AI" since forever. Actually people did this before software is even a thing.

manojlds 3y ago
Exactly. GenAI is much closer to AI than whatever people were calling as AI before.
- namaria 3y ago
  It looks like advances in software asymptotically approximate intelligent seeming behavior, and I think we have come as close as at infinity as the human economy can provide energy for.

numpad0 3y ago

"Training" is an even better one. Anthropomorphizing algorithm behaviors was possibly the biggest by social impact innovation in AI in past 5+ years.

krapp 3y ago

The real marketing is in the constant refrain that LLM hallucinations are precisely equivalent to common human behavior, and that LLMs are thus no less reliable or accurate than human beings, in order to normalize hallucination as an acceptable (and unavoidable) consequence of integrating LLMs into our workflows. That despite every example of an LLM failing in production giving results no competent human would actually produce unless they were committing fraud or had severe mental deficiency, that and no one else would find acceptable.

ben_w 3y ago
> the constant refrain that LLM hallucinations are precisely equivalent to common human behavior You sure it's "precisely" and not "analogously"? Ironically, one common human failing is to use binary classification. > or had severe mental deficiency It seems I have worked with more human idiots than you. And at least one time where I was the idiot, despite literally scoring "off the charts" on a cognitive abilities test at school. I'd agree that LLMs are not as smart as humans — they had to read 10% of the internet just to be as competent as an intern — but to me they're more like newspapers and the Gell-Mann Amnesia effect than "severely deficient", even when indeed unacceptable.
- daveguy 3y ago
  > It seems I have worked with more human idiots than you. This is completely unnecessary.
  - ben_w 3y ago
    That sounds like you think I'm calling @krapp a member of the set "idiots"; I'm not — ironic that at least one of the two of us is misunderstanding the other, given the question of LLMs not understanding things — rather I'm saying there are many temporarily-idiotic in the world (myself included :P) and I have worked with enough of them to not dismiss the mistakes that LLMs make as a dramatic departure from this.
    - 3y ago

rauhl 3y ago

I forget where I first saw it, but I’ve been persuaded that the correct word is ‘confabulation,’ not ‘hallucination.’ Maybe it was this paper: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10619792/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10619792/ The authors’ statement that ‘unlike hallucinations, confabulations are not perceived experiences but instead mistaken reconstructions of information which are influenced by existing knowledge, experiences, expectations, and context’ is pretty compelling.