3 ms·
Aren't those saying the same thing? .. "its reliability became close to none" vs. "causes irreversible defects in the resulting models"
by eikenberry 2y ago
Aren't those saying the same thing? .. "its reliability became close to none" vs. "causes irreversible defects in the resulting models"
- renewiltord 2y agoThere’s thing that happens in humans called “hallucination” where they just make up stuff. You can’t really just take it at face value. Sometimes, they’re just overfit so they generate the same tokens independent of input.
- Guthur 2y agoThat has never been the definition of hallucination, until LLMs. It's actually just called lying, dishonesty or falsehood. Hallucination is a distortion (McKenna might say liberation) of perception. If I hallucinate being covered in spiders, I don't necessarily go around saying, "I'm covered in spiders, if you cant see them you're blind" (disclaimer: some might, but that's not a prerequisite of an hallucination). The cynic in me thinks that use of the word hallucination is marketing to obscure functional inadequacy and reinforce the illusion that LLMs are some how analogous to human intelligence.
- radlad 2y agoLying, dishonesty, and falsehood all imply motive/intent, which is not likely the case when referring to LLM hallucinations. Another term is "making a mistake," but this also reinforces the similarities between humans and LLMs, and doesn't feel very accurate when talking about a technical machine. Sibling commenter correctly calls out the most similar human phenomenon: confabulation ("a memory error consisting of the production of fabricated, distorted, or misinterpreted memories about oneself or the world" per Wikipedia.)
- usual_user 2y agoIMHO Lying means thinking one thing and saying another, hiding your true internal state. For it to be effective it also seems to require something like “theory of mind” (what does the other person know / think that I know).
- jfim 2y agoI believe the term hallucination comes from vision models, where the model would "hallucinate" an object where none exists.
- alcover 2y agoWouldn't 'illusion' be the more precise term here ? When one thinks he recognizes something, whereas 'hallucination' is more of unreal appearing out of the blue ?
- dijksterhuis 2y agoYou might be thinking of deepdream... https://en.wikipedia.org/wiki/DeepDream https://en.wikipedia.org/wiki/DeepDream "Hallucinations" have only really been a term of art with regards to LLMs. my PhD in the security of machine learning started in 2019 and no-one ever used that term in any papers. the first i saw it was on HN when ChatGPT became a released product. Same with "jailbreaking". With reference to machine learning models, this mostly came about when people started fiddling with LLMs that had so-called guardrails implemented. "jailbreaking" is just another name for an adversarial example (test-time integrity evasion attack), with a slightly modified attacker goal.
- alabastervlog 2y ago"Hallucination" is what LLMs are always doing. We only name it that when what they imagine doesn't match reality as well as we'd like, but it's all the same.
- goatlover 2y agoConfabulation is the accurate psychological term. Hallucination is a perceptual issue. The LLM term is misleading.
- moffkalast 2y agoConfabulation would be actual false memories no? I suppose some of it are consistent false beliefs, but more often than not it's well.. lapsus. One token gets generated wrong, or the sampler picks something mindbogglingly dumb that doesn't make any sense because of high temperature, and the model can't help but try and continue as confidently as it can, pretending everything is fine without any option to correct itself. Some thinking models can figure these sort of mistakes out in the long run but it's still not all that reliable and requires training it that way from the base model. Confident bullshitting seems to be very ingrained in current instruct datasets.
- ben_w 2y agoGetting high on their own supply. It's a problem when humans do this. That AI also do it… is interesting… but AI's failure is not absolved by human failure.
- qwertox 2y agoI think parent may have understood it as "second to none", as in "exceptional". That is at least the way I struggled with that sentence in the paper.