8 ms·
> It's still a big issue that the models will make up plausible sounding but wrong or misleading explanations for things, and verifying their claims ends up tak
by stacktrace 10mo ago
> It's still a big issue that the models will make up plausible sounding but wrong or misleading explanations for things, and verifying their claims ends up taking time. And if it's a topic you don't care about enough, you might just end up misinformed.
Exactly! One important thing LLMs have made me realise deeply is "No information" is better than false information. The way LLMs pull out completely incorrect explanations baffles me - I suppose that's expected since in the end it's generating tokens based on its training and it's reasonable it might hallucinate some stuff, but knowing this doesn't ease any of my frustration.
IMO if LLMs need to focus on anything right now, they should focus on better grounding. Maybe even something like a probability/confidence score, might end up experience so much better for so many users like me.
- robocat 10mo ago> wrong or misleading explanations Exactly the same issue occurs with search. Unfortunately not everybody knows to mistrust AI responses, or have the skills to double-check information.
- darkwater 10mo agoNo, it's not the same. Search results send/show you one or more specific pages/websites. And each website has a different trust factor. Yes, plenty of people repeat things they "read on the Internet" as truths, but it's easy to debunk some of them just based on the site reputation. With AI responses, the reputation is shared with the good answers as well, because they do give good answers most of the time, but also hallucinate errors.
- SebastianSosa1 10mo agoCommunity notes on X seems to be one of the highest profile recent experiments trying to address this issue
- dexterlagan 10mo agoMy attempt: https://www.cleverthinkingsoftware.com/truth-or-extinction/ https://www.cleverthinkingsoftware.com/truth-or-extinction/
- darkwater 10mo ago> Tools like SourceFinder must be paired with education — teaching people how to trace information themselves, to ask: Where did this come from? Who benefits if I believe it? These are very important and relevant questions to ask oneself when you read about anything, but we also keep in mind that even those question can be misused and they can drive you to conspiracy theories.
- deleted 10mo ago[deleted]
- incrudible 10mo agoIf somebody asks a question on Stackoverflow, it is unlikely that a human who does not know the answer will take time out of their day to completely fabricate a plausible sounding answer.
- balder1991 10mo agoAt least it used to be true.
- jaxn 10mo agoPeople are confidently incorrect all the time. It is very likely that people will make up plausible sounding answers on StackOverflow. You and I have both taken time out of our days to write plausible sounding answers that are essentially opposing hallucinations.
- linen 10mo agoSites like stackoverflow are inherently peer-reviewed, though; they've got a crowdsourced voting system and comments that accumulate over time. People test the ideas in question. This whole "people are just as incorrect as LLMs" is a poor argument, because it compares the single human and the single LLM response in a vacuum. When you put enough humans together on the internet you usually get a more meaningful result.
- JAlexoid 10mo agoHave you ever heard of Dunning Kruger effect? There's a reason why there are upvotes, solution and third party edit system in StackOverflow - people will spend time to write their "hallucinations" very confidently.
- lins1909 10mo agoWhat is it about people making up lies to defend LLMs? In what world is it exactly the same as search? They're literally different things, since you get information from multiple sources and can do your own filtering.
- actionfromafar 10mo agoI wonder if the only way to fix this with current LLMs, would be to generate a lot synthetic data for a select number topics you really don't want it "go off the rails" with. That synthetic data would be lots of variations on that "I don't know how to do X with Y".
- dolmen 10mo agoI would not bet on synthetic data. LLMs are very good at detecting patterns.
- XCSme 10mo agoBut most benchmarks are not about that... Are there even any "hallucination" public benchmarks?
- biofox 10mo agoI ask for confidence scores in my custom instructions / prompts, and LLMs do surprisingly well at estimating their own knowledge most of the time.
- drclau 10mo agoHow do you know the confidence scores are not hallucinated as well?
- dfsegoat 10mo agothey 100% are unless you provide a RUBRIC / basically make it ordinal. "Return a score of 0.0 if ...., Return a score of 0.5 if .... , Return a score of 1.0 if ..."
- kiliankoe 10mo agoThey are, the model has no inherent knowledge about its confidence levels, it just adds plausible-sounding numbers. Obviously they _can_ be plausible, but trusting these is just another level up from trusting the original output. I read a comment here a few weeks back that LLMs always hallucinate, but we sometimes get lucky when the hallucinations match up with reality. I've been thinking about that a lot lately.
- TeMPOraL 10mo ago> the model has no inherent knowledge about its confidence levels Kind of. See e.g. https://openreview.net/forum?id=mbu8EEnp3a https://openreview.net/forum?id=mbu8EEnp3a, but I think it was established already a year ago that LLMs tend to have identifiable internal confidence signal; the challenge around the time of DeepSeek-R1 release was to, through training, connect that signal to tool use activation, so it does a search if it "feels unsure".
- losvedir 10mo agoWow, that's a really interesting paper. That's the kind of thing that makes me feel there's a lot more research to be done "around" LLMs and how they work, and that there's still a fair bit of improvement to be found.
- basisword 10mo agoI think the thing even worse than false information is the almost-correct information. You do a quick Google to confirm it's on the right page but find there's an important misunderstanding. These are so much harder to spot I think than the blatantly false.
- RHSman2 10mo agoThe problem is not the intelligence of the LLM. It is the intelligence and desire to make things easy of the intelligence using them.