2 ms·
You can argue that everything output by an LLM is hallucinated, since there's no difference under-the-hood between outputting useful information and outputting
by simonw 2y ago
You can argue that everything output by an LLM is hallucinated, since there's no difference under-the-hood between outputting useful information and outputting hallucinations.
The quality of the LLM then becomes how often it produces useful information. That score has gone up a lot in the past 18 months.
(Sometimes hallucinations are what you want: "Tell me a fun story about a dog learning calculus" is a valid prompt which mostly isn't meant to produce real facts about the world")
- codr7 2y agoIsn't it the case that the latest models actually hallucinate more than the ones that came before? Despite best efforts to prevent it.
- simonw 2y agoThe o3 model card reports a so far unexplained uptick in hallucination rate from o1 - on page 4 of https://cdn.openai.com/pdf/2221c875-02dc-4789-800b-e7758f3722c1/o3-and-o4-mini-system-card.pdf https://cdn.openai.com/pdf/2221c875-02dc-4789-800b-e7758f372... That is according to one specific internal OpenAI benchmark, I don't know if it's been replicated externally yet.