3 ms·
"Hallucinate" predates LLMs. For instance, before transformers, when we finetuned ESRGAN for specific styles/functions, we used to say it would "hallucinate" i
by brucethemoose2 3y ago
"Hallucinate" predates LLMs.
For instance, before transformers, when we finetuned ESRGAN for specific styles/functions, we used to say it would "hallucinate" in detail that couldn't possibly be in the original image pixels. GenAI wasn't much of a thing back then, and we never thought of the small GAN models as "remembering" things even though thats common language in transformer models. It felt more like we were teaching the GAN a job or style, not encoding memory.
And I think it fit! "Hallucinating" detail into a blurry pixel blob feels like a more accurate analogy to the human condition, and we weren't at the point where it was "confabulating" and generating big objects out of the blue like a diffusion model can.
- panagathon 3y agoI'm inclined, perhaps mistakenly, to think of it as similar to how our brains fill in our peripheral vision, to respond to the lack of data. https://neurosciencenews.com/peripheral-vision-brain-illusion-5713/ https://neurosciencenews.com/peripheral-vision-brain-illusio...
- swyx 3y agokarpathy says may have coined it in his RNN post https://twitter.com/DrJimFan/status/1703072983903060260 https://twitter.com/DrJimFan/status/1703072983903060260
- matusp 3y agoHe most certainly didn't. I am pretty sure it was used in ML in early 2000s, especially in vision.
- nsagent 3y agohe definitely isn't the first one to use the term in that capacity. it's frequently used by cv researchers in relation to super resolution (e.g. just search "face hallucination" on google scholar) see the abstract of this paper for example Face hallucination using Bayesian global estimation and local basis selection > ...hallucinating the high-resolution detail of a low-resolution input face image. while that's a different domain than text, in a quick search, the oldest reference i found to "hallucination" using vector-based word representations is from 1996 (there could easily be older references, i just didn't bother searching further): Text Databases and Information Retrieval https://dl.acm.org/doi/pdf/10.1145/234313.234371 https://dl.acm.org/doi/pdf/10.1145/234313.234371 > The fixed-length vector is especially useful for parallel and hardware systems, but this method can sometimes hallucinate words that don't actually appear in the original document. edit: including a link to the face hallucination paper i mentioned (there are lots more) https://www.semanticscholar.org/paper/Face-hallucination-using-Bayesian-global-estimation-Hsu-Lin/a0298909a825f3b2aabfe9e2ad251318544d12b5 https://www.semanticscholar.org/paper/Face-hallucination-usi...
- stared 3y agoSure. Yet, there were "deep dreams" to generate "unreal" images back then. In my opinion, these are hallucinations - things that happen in dreams (or under the influence of various substances). See https://blog.research.google/2015/06/inceptionism-going-deeper-into-neural.html https://blog.research.google/2015/06/inceptionism-going-deep... and https://distill.pub/2017/feature-visualization/ https://distill.pub/2017/feature-visualization/. I would differentiate these dreamy generations (i.e., things that we don't see ordinarily) from generations that are incorrect. Sure, there is overlap.
- vanderZwan 3y agoRight, so basically "hallucinate" fits better for visual generative ML and "confabulate" for text. Seems like a sensible distinction to me.