4 ms·
The 120B model badly hallucinates facts on the level of a 0.6B model. My go to test for checking hallucinations is 'Tell me about Mercantour park' (a national
by dust42 1y ago
The 120B model badly hallucinates facts on the level of a 0.6B model.
My go to test for checking hallucinations is 'Tell me about Mercantour park' (a national park in south eastern France).
Easily half of the facts are invented. Non-existing mountain summits, brown bears (no, there are none), villages that are elsewhere, wrong advice ('dogs allowed' - no they are not).
- hmottestad 1y agoI don’t think they trained it for fact retrieval. Would probably do a lot better if you give it tool access for search and web browsing.
- lukev 1y agoThis is precisely the wrong way to think about LLMs. LLMs are never going to have fact retrieval as a strength. Transformer models don't store their training data: they are categorically incapable of telling you where a fact comes from. They also cannot escape the laws of information theory: storing information requires bits. Storing all the world's obscure information requires quite a lot of bits. What we want out of LLMs is large context, strong reasoning and linguistic facility. Couple these with tool use and data retrieval, and you can start to build useful systems. From this point of view, the more of a model's total weight footprint is dedicated to "fact storage", the less desirable it is.
- superconduct123 1y agoHow can you reason correctly if you don't have any way to know which facts are real vs hallucinated?
- futureshock 1y agoI think that sounds very reasonable, but unfortunately these models don’t know what they know and don’t. A small model that knew the exact limits of its knowledge would be very powerful.
- energy123 1y agoHallucinations have characteristics in interpretability studies. That's a foothold into reducing them. They still won't store much information, but it could mean they're better able to know what they don't know.
- CrackerNews 1y agoWhat are the large context, strong reasoning, and linguistic facility for if there aren't facts underpinning them? Is a priori wholly independent of a posteriori? Is it practical for the former to be wholly independent of the latter?
- pocketarc 1y agoOthers have already said it, but it needs to be said again: Good god, stop treating LLMs like oracles. LLMs are not encyclopedias. Give an LLM the context you want to explore, and it will do a fantastic job of telling you all about it. Give an LLM access to web search, and it will find things for you and tell you what you want to know. Ask it "what's happening in my town this week?", and it will answer that with the tools it is given. Not out of its oracle mind, but out of web search + natural language processing. Stop expecting LLMs to -know- things. Treating LLMs like all-knowing oracles is exactly the thing that's setting apart those who are finding huge productivity gains with them from those who can't get anything productive out of them.
- diegocg 1y agoThe problem is that even when you give them context, they just hallucinate at another level. I have tried that example of asking about events in my area, they are absolutely awful at it.
- Salgat 1y agoIt's fine to expect it to not know things, but the complaint is that it makes zero indication that it's just making up nonsense, which is the biggest issue with LLMs. They do the same thing when creating code.
- dust42 1y agoExactly this. And that is why I like this question because the amount of correct details and the amount of nonsense give a good idea about the quality of the model.
- dankwizard 1y agoI love how with this cutting edge tech people still dress up and pretend to be experts. Pleasure to meet you, pocketarc - Senior AI Gamechanger, 2024-2025 (Current)
- saurik 1y agoI am getting huge productivity gains from using models, and I mostly use them as "oracles" (though I am extremely careful with respect to how I have to handle hallucination, of course): I'd even say their true power--just like a human--comes from having an ungodly amount of knowledge, not merely intelligence. If I just wanted something intelligent, I already had humans!... but merely intelligent humans, even when given months of time to screw around doing Google searches, fail to make the insights that someone--whether they are a human or a model--that actually knows stuff can throw around like it is nothing. I am actually able to use ChatGPT 4.5 as not just an employee, not even just as a coworker, but at times as a mentor or senior advisor: I can tell it what I am trying to do, and it helps me by applying advanced mathematical insights or suggesting things I could use. Using an LLM as a glorified Google-it-for-me monkey seems like such a waste of potential.