4 ms·
> hallucinates and generates invalid data The model is doing the exact same thing when it generates "correct" output as it does when it generates "incorrect" o
by SrslyJosh 2y ago
> hallucinates and generates invalid data
The model is doing the exact same thing when it generates "correct" output as it does when it generates "incorrect" output.
"Hallucination" is a misleading term, cooked up by people who either don't understand what's going on or who want to make it sound like the fundamental problems (models aren't intelligent, can't reason, and attach zero meaning to their input or output) can be solved with enough duct tape.
- Terr_ 2y agoI like the analogy of an Ouija board: "This magical conduit to supernatural forces sometimes gives out nonsense, but soon we'll fix it so that it always channels the correct ghosts and spirits."
- RogerL 2y agoIt was "cooked up" by people like Joshua Maynez, Shashi Narayan, et al, who I'm going to guess understand what is going on, and adopted by the rest of the field. https://aclanthology.org/2020.acl-main.173/ https://aclanthology.org/2020.acl-main.173/
- troupo 2y agoI like the term because it's concise and describes the output :) But true, it's just the output.