3 ms·
What does it mean to get something right? If I ask GPT-4o (or any model, really) whether it's usually hot in Topeka during the summer, it will give me the corre
by mrtranscendence 2y ago
What does it mean to get something right? If I ask GPT-4o (or any model, really) whether it's usually hot in Topeka during the summer, it will give me the correct answer. The model may or may not understand what it means for Topeka to be hot during the summer, but it has generated text with a single reasonable interpretation which happens to include a factually correct statement. I'm comfortable saying that GPT-4o "got it right" in that case regardless of what we believe about its understanding.
- glenstein 2y agoI don't think anyone concerned about hallucinations is talking about broad brush statements which can be true generally speaking. It's about much more literal facts like code calling non-existent functions or citing non-existent court cases. Or, in my case yesterday, telling me a particular Nextcloud app could edit audio meta-data, when it in fact cannot. There are grey areas about what it means to be right to be sure, but there are also black and white areas, and it's the latter where the hallucinations are happening.