7 ms·
I wonder how much the source content is the cause of hallucinations rather than anything inherent to LLMs. I mean if someone posts a question on an internet fo
by aidenn0 3y ago
I wonder how much the source content is the cause of hallucinations rather than anything inherent to LLMs. I mean if someone posts a question on an internet forum that I don't know the answer to, I'm certainly not going to post "I don't know" since that wouldn't be useful.
In fact, in general, in any non one-on-one conversation the answer "I don't know" is not useful because if you don't know in a group, your silence indicates that.
- abracadaniel 3y agoThat’s a good observation. If LLMs had taken off 15 years ago, maybe they would answer every question with “this has already been asked before. Please use the search function”
- exe34 3y agoMarked as duplicate.
- btown 3y agoYour question may be a better fit for a different StackExchange site.
- amadeuspagel 3y agoWe prefer questions that can be answered, not merely discussed.
- exe34 3y agoFeel like we're going to get dang on our case soon...
- TeMPOraL 3y agoTopic locked as not constructive.
- taco-hands 3y agoDamn. I was hoping to be the fastest gun in the west on this one!
- SirMaster 3y agoI thought LLMs don't say when they don't know something because of how they are tuned and because of RLHF.
- ben_w 3y agoThey can say they don't know, and have been trained to in at least some cases; I think the deeper problem — which we don't know how to fix in humans, the closest we have is the scientific method — is they can be confidently wrong.
- jdashg 3y agoNowadays they are instead learning to say "please join our Discord for support"!
- TeMPOraL 3y agoMuch like one of the first phrases spoken by babies today is "like and subscribe".
- Voultapher 3y agoPart of me really wants to believe this is a joke. But given how often toddlers are "taken care of" by planting them in front of youtube :|
- jvanderbot 3y agoWhich is crazy because there's plenty of good content for kids on Youtube (if you really need a break!). Blippy, Meekah, Seasame Street, even that mind-numbing drivel Cocomelon (which at least got my girls talking/singing really early).
- realfeel78 3y agoThere's actually no such thing as good "content" for kids, sorry.
- jvanderbot 3y agoI get the sentiment, but when reality hits unrealistic parental expectations, things get messy. If you have to put a show on TV to give some songs to sing along to or to distract them while you're making lunch, I'm not judging you, and I think it's best to put this content on a gradient rather than black and white.
- 9991 3y agoFor all of human history until 70 years ago, no baby watched TV. Reconsider what you "have to" do.
- darby_eight 3y agoWith Wittgenstein I think we see that "hallucinations" are a part of language in general, albeit one I could see being particularly vexing if you're trying to build a perfectly controllable chatbot.
- Y_Y 3y agoThis sounds interesting, could you give more detail on what you're referring to?
- all2 3y agoI would assume GP is talking about the fallibility of human memory, or perhaps about the meanings of words/phrases/aphorisms that drift with time. C.S. Lewis talks about the meaning of the word "gentleman" in one of his books; at first the word just meant "land owner" and that was it. Then it gained social significance and began to be associated with certain kinds of behavior. And now, in the modern register, its meaning is so dilute that it can be anything from "my grandson was well behaved today" or "what an asshole" depending on its use context. Dunno. GP?
- darby_eight 3y agoI'm referring to his two works, the "Tractatus Logico-Philosophicus" and "Philosophical Investigations". There's a lot explored here, but Wittgenstein basically makes the argument that the natural logic of language—how we deduce meaning from terms in a context and naturally disambiguate the semantics of ambiguous phrases—is different from the sort of formal propositional logic that forms the basis of western philosophy. However, this is also the sort of logic that allows us to apply metaphors and conceive of (possibly incoherent, possibly novel, certainly not deductively-derived) terms—counterfactuals, conditionals, subjunctive phrases, metaphors, analogies, poetic imagery, etc. LLMs have shown some affinity of the former (linguistic) type of logic with greatly reduced affinity with the latter (formal/propositional) sort of logical processing. Hallucinations as people describe them seem to be problems with not spotting "obvious" propositional incoherence. What I'm pushing at is not that this linguistic ability naturally leads to the LLM behavior we're seeing and calling "hallucinating", just that LLMs may capture some of how humans process language, differentiate semantics, recall terms, etc, but without the mechanisms that enable rationally grappling with the resulting semantics and propositional (in)coherency that are fetched or generated. I can't say this is very surprising—most of us seem to have thought processes that involve generating and rejecting thoughts when we e.g. "brainstorm" or engage in careful articulation that we haven't even figured out how to formally model with a chatbot capable of generating a single "thought", but I'm guessing if we want chatbots to keep their ability to generate things creatively there will always be tension with potentially generating factual claims, erm, creatively. Further evidence is anecdotal observations that some people seem to have wildly different thresholds for the propositional coherence they can spot—perhaps one might be inclined to correlate the complexity with which one can engage in spotting (in)coherence with "intelligence", if one considers that a meaningful term.
- shkkmo 3y ago>In fact, in general, in any non one-on-one conversation the answer "I don't know" is not useful because if you don't know in a group, your silence indicates that. This isn't true. There are many contexts where it is true but it doesn't actually generalize they way you say it does. There are plenty of cases where experts in a non-one-on-one context will express a lack of knowledge. Sometimes this will be as part of making point about the broader epistemic state of the group, sometimes it will be simply to clarify the epistemic state of the speaker.
- lanstin 3y agoI personally will almost always say I don't know while talking thru to a solution. Admittedly this is informal speech that doesn't make it to written form.
- digging 3y ago> I wonder how much the source content is the cause of hallucinations rather than anything inherent to LLMs I mean, it's inherent to LLMs to be unable to answer "I don't know" as a result of not knowing the answer. An LLM never "doesn't know" the answer. But they'll gladly answer "I don't know" if that's statistically the most likely response, right? (Although current public offerings are probably trained against ever saying that.)
- aidenn0 3y agoLLMs work at all because of the high correlation between the statistically most likely response and the most reasonable answer.
- digging 3y agoThat's an explanation of why their answers can be useful, but doesn't relate to their ability to "not know" an answer
- ben_w 3y agoI suspect this is going to be a disagreement on the meaning of "to know". On the same lines as why people argue if a tree falling in a wood where nobody can hear it makes sound because some people implicitly regard sound is the qualia while others regard it as the vibrations in the air.
- digging 3y agoNot really. An LLM should have no problem replying "I don't know" if that's the most statistically likely answer to a given question, and if it's not trained against such a response. What it fundamentally can't do is introspect and determine it doesn't have enough information to answer the question. It always has an answer. (disclaimer: I don't know jack about the actual mechanics. It's possible something could be constructed which does have that ability and still be considered an "LLM". But the ones we have now can't do that.)
- 3y ago
- CuriouslyC 3y agoA lot of LLM hallucination is because of the internal conflict between alignment for helpfulness and lack of a clear answer. It's much like when someone gets out of their depth in a conversation and dissembles their way through to try and maintain their illusion of competence. In these cases, if you give the LLM explicit permission to tell you that it doesn't know in cases where it's not sure, that will significantly reduce hallucinations. A lot more of LLM hallucination is it getting the context confused. I was able to get GPT4 to hallucinate easily with questions related to the distance from one planet to another, since most distances on the internet are from the sun to individual planets, and the distances between planets varies significantly based on their locations in cycle. These are probably slightly harder to fix.
- danenania 3y ago"In these cases, if you give the LLM explicit permission to tell you that it doesn't know in cases where it's not sure, that will significantly reduce hallucinations." I've noticed that while this can help to prevent hallucinations, it can also cause it to go way too far in the other direction and start telling you it doesn't know for all kinds of questions it really can answer.
- sumtechguy 3y agoMy current favorite one is to ask the time. Then ask it if it is possible for it to give you the time. You get 2 very different answers.
- mzi 3y agoIt also has a problem with quantity, so it gets confused by things like the cube root of 750l that it maintains for a long time is around 9m. It even suggests that 1l is equal to 1m³.
- hiatus 3y agoContrast with Q&A on products on Amazon where people routinely answer that way. I have flagged responses saying "I don't know" but nothing ever comes of it.
- aendruk 3y agoI’d place in the same category the responses that I give to those chat popups so many sites have. They show a person saying to me “Can I help you with anything today?” so I always send back “No”.
- ceejayoz 3y agoThis is Amazon's fault; they send an email that looks like it's specifically directed to you. "ceejayoz, a fellow customer has a question on..." At some point fairly recently they added a "I don't know the answer" button to the email, but it's much less prominent than the main call-to-action.
- philipswood 3y agoI've wondered if one could train a LLM on a closed set of curated knowledge. Then include training data that models the behaviour of not knowing. To the point that it could generalize to being able to represent its own not knowing. Because expecting a behaviour, like knowing you don't know, that isn't represented in the training set is silly. Kids make stuff up at first, then we correct them - so they have a way to learn not to.
- aidenn0 3y ago> I've wondered if one could train a LLM on a closed set of curated knowledge. Then include training data that models the behaviour of not knowing. To the point that it could generalize to being able to represent its own not knowing. The problem is that curating data is slow and expensive and downloading the entire web is fast and cheap. See also https://en.wikipedia.org/wiki/Cyc https://en.wikipedia.org/wiki/Cyc
- philipswood 3y agoAgreed. Using LLM to generate or curate training sets for other generations seems like a cool approach. Maybe if you trained a small base model to know it doesn't know in general and THEN trained it on the entire web with embedded not-knowing preserving training examples, it would work?
- philipswood 3y agoReminded of this approach where a tiny model was trained children's stories generated by a larger model: https://www.quantamagazine.org/tiny-language-models-thrive-with-gpt-4-as-a-teacher-20231005/ https://www.quantamagazine.org/tiny-language-models-thrive-w...
- crooked-v 3y ago> train a LLM a closed set of curated knowledge Google has one of these already, with an LLM that was trained on nothing but weather data and so can only give weather-data-prediction responses. The 'knowing it doesn't know things' part is much harder to get reliable, though.
- Rebelgecko 3y agoReminds me of a joke Three logicians walk into a bar. The bartender says "what'll it be, three beers?" The first logician says "I don't know". The second logician says "I don't know". The third logician says "Yes".
- withinboredom 3y agoAnd the human bartender passes the check to the third logician.
- readyplayernull 3y agoThe third logician never finishes his beer, his friends get more free beers. The bar overflows.
- dylan604 3y agoHow much would that exploit be worth on the open market?
- withinboredom 3y agoIt's one you keep to yourself and your friends. Otherwise, it will get fixed up.
- deleted 3y ago[deleted]
- mgsouth 3y agoIf, like me, you didn't get the joke at first: Both of the first two logicians wanted a beer; otherwise they would know the answer was "no". The third logician recognizes this, and therefore knows the answer.
- KptMarchewa 3y agoUnless one of those wanted two beers. Or 0.5 beer. Or -1 beers. Or 1e9 beers. Or 2147483648 beers.
- troq13 3y ago"I wonder how much the source content is the cause of hallucinations rather than anything inherent to LLMs." Probably true, but if you have quality, organized data, you will just want to search the data itself.
- singingfish 3y agoI'm really cross that the word "hallucination" has taken off to describe this as it's clearly in incorrect word. The correct word to describe it is "confabulation", which is clinically more accurate and a much clearer descriptor of what's actually going on. https://en.wikipedia.org/wiki/Confabulation https://en.wikipedia.org/wiki/Confabulation
- mindcrime 3y agoI proposed[1] the portmanteau "hallucofabulation" as a compromise, but it hasn't caught on yet. I'm totally shocked and dismayed by this, of course. [1]: https://news.ycombinator.com/item?id=36977935 https://news.ycombinator.com/item?id=36977935
- brookst 3y agoThe re-use of the "c" as a soft c in "hallucinate" and then a hard c in confabulate is confusing, and probably affecting the uptake of your neologism.
- fhars 3y agoYes, it would probably have to be halucinfabulation for purely phonetic reasons.
- mindcrime 3y agoMaybe if I added a hyphen? "halluco-fabulation"?
- singingfish 3y agowould lead to hafabulation. Still not acceptable in my view - confabulation is clearly the correct term
- mcmcmc 3y agoIt would probably help if you didn't drop the "n" from the confabulation part.
- Traubenfuchs 3y ago> In fact, in general, in any non one-on-one conversation the answer "I don't know" is not useful because if you don't know in a group, your silence indicates that. Only a response makes it clear one has read and acknowledged the question and sometimes there are people expected to know, if they don‘t the should say so.