3 ms·

The best trick the A.I. companies have pulled is getting us to refer to ‘bugs’ as ‘hallucinations.’ It sounds so much more sophisticated.

dgellow 2y ago
It’s not a trick to sound sophisticated. Hallucinations are more like a subcategory of bugs. The system is technically correctly generating, structuring, and presenting false information as fact.
- greg_V 2y ago
  Technically everything an LLM does is hallucination that happens to be on a scale between correct and non-correct. But only humans with knowledge can tell the difference, math alone can't. It's not even a bug: it's the defining feature of the technology!
  - elwell 2y ago
    > But only humans with knowledge can tell the difference Who says the humans (all of them) aren't hallucinating too?
  - astrange 2y ago
    Knowledge isn't sufficient to show something is false, since the knowledge can also be false. Insofar as it's important for it to be true, it needs to be continually verified as true, so that it's grounded in the real world.
chx 2y ago
Ah, my friend it's not a bug It's a fundamental feature These LLMs can produce nothing else but since the bullshit they spew resembles an answer and sometimes accidentally collide with one, people tend to think it can give answers. But no. https://hachyderm.io/@inthehands/112006855076082650 https://hachyderm.io/@inthehands/112006855076082650 > You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, for having something crappy to react to. > Alas, that does not remotely resemble how people are pitching this technology.
- kwertyoowiyop 2y ago
  That’s a good take. So LLMs distill human creativity as well as human knowledge, and it’s more useful when their creativity goes off the rails than when their knowledge does.
- astrange 2y ago
  This is irrelevant because the LLM is mostly not answering the question directly, it's summarizing text from web results. Quoting a joke isn't a hallucination.

dgellow 2y ago

It’s not a trick to sound sophisticated. Hallucinations are more like a subcategory of bugs. The system is technically correctly generating, structuring, and presenting false information as fact.

greg_V 2y ago
Technically everything an LLM does is hallucination that happens to be on a scale between correct and non-correct. But only humans with knowledge can tell the difference, math alone can't. It's not even a bug: it's the defining feature of the technology!
- elwell 2y ago
  > But only humans with knowledge can tell the difference Who says the humans (all of them) aren't hallucinating too?
- astrange 2y ago
  Knowledge isn't sufficient to show something is false, since the knowledge can also be false. Insofar as it's important for it to be true, it needs to be continually verified as true, so that it's grounded in the real world.

chx 2y ago

Ah, my friend it's not a bug It's a fundamental feature These LLMs can produce nothing else but since the bullshit they spew resembles an answer and sometimes accidentally collide with one, people tend to think it can give answers. But no. https://hachyderm.io/@inthehands/112006855076082650 https://hachyderm.io/@inthehands/112006855076082650 > You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, for having something crappy to react to. > Alas, that does not remotely resemble how people are pitching this technology.

kwertyoowiyop 2y ago
That’s a good take. So LLMs distill human creativity as well as human knowledge, and it’s more useful when their creativity goes off the rails than when their knowledge does.
astrange 2y ago
This is irrelevant because the LLM is mostly not answering the question directly, it's summarizing text from web results. Quoting a joke isn't a hallucination.