5 ms·

Why do people act like LLMs only hallucinate some of the time?

kwertyoowiyop 2y ago
The best trick the A.I. companies have pulled is getting us to refer to ‘bugs’ as ‘hallucinations.’ It sounds so much more sophisticated.
- dgellow 2y ago
  It’s not a trick to sound sophisticated. Hallucinations are more like a subcategory of bugs. The system is technically correctly generating, structuring, and presenting false information as fact.
  - greg_V 2y ago
    Technically everything an LLM does is hallucination that happens to be on a scale between correct and non-correct. But only humans with knowledge can tell the difference, math alone can't. It's not even a bug: it's the defining feature of the technology!
    - elwell 2y ago
      > But only humans with knowledge can tell the difference Who says the humans (all of them) aren't hallucinating too?
    - astrange 2y ago
      Knowledge isn't sufficient to show something is false, since the knowledge can also be false. Insofar as it's important for it to be true, it needs to be continually verified as true, so that it's grounded in the real world.
- chx 2y ago
  Ah, my friend it's not a bug It's a fundamental feature These LLMs can produce nothing else but since the bullshit they spew resembles an answer and sometimes accidentally collide with one, people tend to think it can give answers. But no. https://hachyderm.io/@inthehands/112006855076082650 https://hachyderm.io/@inthehands/112006855076082650 > You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, for having something crappy to react to. > Alas, that does not remotely resemble how people are pitching this technology.
  - kwertyoowiyop 2y ago
    That’s a good take. So LLMs distill human creativity as well as human knowledge, and it’s more useful when their creativity goes off the rails than when their knowledge does.
  - astrange 2y ago
    This is irrelevant because the LLM is mostly not answering the question directly, it's summarizing text from web results. Quoting a joke isn't a hallucination.
dgellow 2y ago
It’s not hallucinations here, multiple of the ridiculous results can be directly traced to redit posts where people are joking or saying absurd things
- threeseed 2y ago
  There are examples of hallucinations as well e.g. talking about a Google AI dataset that doesn't exist and using a CSAM dataset which it doesn't. One of the researchers from Google Deepmind specifically said it was hallucinating.
- deleted 2y ago
  [deleted]
- notfed 2y ago
  So...every Reddit post?
notnullorvoid 2y ago
Hmm yeah I kinda like the concept that it's "hallucinating" 100% of the time, and it just so happens that x% of those hallucinations accurately describe the real world.
- empath75 2y ago
  That x% is far higher than people think it is because there's a tremendous amount of information about the world that ai models need to "understand" that people just kind of take for granted and don't even think about. A couple of years ago, AI's routinely got "the basics" wrong, but now so often get most things right that people don't even think it's worth commenting on that they do. In any case, human consciousness is also a hallucination.
  - notnullorvoid 2y ago
    It really depends on the set of prompts you present the LLM. If it's anything requiring reasoning, you'll often get nonsense that sounds like sense. It has a higher chance of being accurate with knowledge queries. LLMs are impressive, a very lossy search engine in a small package, capable of outputting convincing natural language responses.
itronitron 2y ago
it's only AI if you believe it
kccqzy 2y ago
Not hallucinations but these AI answers often (always?) provide sources they link to. It's just that the source is a random Reddit or Quora post that's obviously just trolling. Then, when people post these weird AI answers on Reddit and come up with more absurd jokes, the AI then picks it up again. For example in https://www.reddit.com/r/comedyheaven/comments/1cq4ieb/food_names_end_with_um/ https://www.reddit.com/r/comedyheaven/comments/1cq4ieb/food_... Google AI suggested applum and bananum as a response to food names ending with "um" when someone suggested uranium, Copilot AI started copied that suggestion. It's entertaining to watch.

The best trick the A.I. companies have pulled is getting us to refer to ‘bugs’ as ‘hallucinations.’ It sounds so much more sophisticated.

dgellow 2y ago
It’s not a trick to sound sophisticated. Hallucinations are more like a subcategory of bugs. The system is technically correctly generating, structuring, and presenting false information as fact.
- greg_V 2y ago
  Technically everything an LLM does is hallucination that happens to be on a scale between correct and non-correct. But only humans with knowledge can tell the difference, math alone can't. It's not even a bug: it's the defining feature of the technology!
  - elwell 2y ago
    > But only humans with knowledge can tell the difference Who says the humans (all of them) aren't hallucinating too?
  - astrange 2y ago
    Knowledge isn't sufficient to show something is false, since the knowledge can also be false. Insofar as it's important for it to be true, it needs to be continually verified as true, so that it's grounded in the real world.
chx 2y ago
Ah, my friend it's not a bug It's a fundamental feature These LLMs can produce nothing else but since the bullshit they spew resembles an answer and sometimes accidentally collide with one, people tend to think it can give answers. But no. https://hachyderm.io/@inthehands/112006855076082650 https://hachyderm.io/@inthehands/112006855076082650 > You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, for having something crappy to react to. > Alas, that does not remotely resemble how people are pitching this technology.
- kwertyoowiyop 2y ago
  That’s a good take. So LLMs distill human creativity as well as human knowledge, and it’s more useful when their creativity goes off the rails than when their knowledge does.
- astrange 2y ago
  This is irrelevant because the LLM is mostly not answering the question directly, it's summarizing text from web results. Quoting a joke isn't a hallucination.

dgellow 2y ago

It’s not hallucinations here, multiple of the ridiculous results can be directly traced to redit posts where people are joking or saying absurd things

threeseed 2y ago
There are examples of hallucinations as well e.g. talking about a Google AI dataset that doesn't exist and using a CSAM dataset which it doesn't. One of the researchers from Google Deepmind specifically said it was hallucinating.
deleted 2y ago
[deleted]
notfed 2y ago
So...every Reddit post?

notnullorvoid 2y ago

Hmm yeah I kinda like the concept that it's "hallucinating" 100% of the time, and it just so happens that x% of those hallucinations accurately describe the real world.

empath75 2y ago
That x% is far higher than people think it is because there's a tremendous amount of information about the world that ai models need to "understand" that people just kind of take for granted and don't even think about. A couple of years ago, AI's routinely got "the basics" wrong, but now so often get most things right that people don't even think it's worth commenting on that they do. In any case, human consciousness is also a hallucination.
- notnullorvoid 2y ago
  It really depends on the set of prompts you present the LLM. If it's anything requiring reasoning, you'll often get nonsense that sounds like sense. It has a higher chance of being accurate with knowledge queries. LLMs are impressive, a very lossy search engine in a small package, capable of outputting convincing natural language responses.

itronitron 2y ago

it's only AI if you believe it

kccqzy 2y ago

Not hallucinations but these AI answers often (always?) provide sources they link to. It's just that the source is a random Reddit or Quora post that's obviously just trolling. Then, when people post these weird AI answers on Reddit and come up with more absurd jokes, the AI then picks it up again. For example in https://www.reddit.com/r/comedyheaven/comments/1cq4ieb/food_names_end_with_um/ https://www.reddit.com/r/comedyheaven/comments/1cq4ieb/food_... Google AI suggested applum and bananum as a response to food names ending with "um" when someone suggested uranium, Copilot AI started copied that suggestion. It's entertaining to watch.