3 ms·
~40% hallucinations on SimpleQA by a frontier reasoner (o1) and a frontier non-reasoner (GPT-4.5). More orders of magnitude in scale isn't going to fix this def
by energy123 2y ago
~40% hallucinations on SimpleQA by a frontier reasoner (o1) and a frontier non-reasoner (GPT-4.5). More orders of magnitude in scale isn't going to fix this deficit. There's something fundamentally wrong with the approach. A human is much more capable of saying "I don't know" in the correct spots, even if a human is also susceptible to false memories.
Probably OpenAI thinks that tool use (search) will be sufficient to solve this problem. Maybe that will be the case.
Are there any creative approaches to fixing this problem?