4 ms·
The problem is that the limits to LLM answers have more dimensions than just "uncertainty". There is "the question/phrase lacks meaning", "I don't have enough i
by joe_the_user 2y ago
The problem is that the limits to LLM answers have more dimensions than just "uncertainty". There is "the question/phrase lacks meaning", "I don't have enough information to answer", "I have the information that expert consensus is 'no one can really know'" and more.
I think there's a human tendency to reduce the problem one has answering a given question to a question of just "uncertainty" and so we look at LLM answers as involving just single level of uncertainty. But that's anthropomorphism.
AI images (and photograph before it) showed us new, unimagined ways an image can be wrong (or rather, real-seaming but wrong). AI language interactions do this too but in a more subtle way.
- trq_ 2y agoDefinitely, but if you can detect when you might be in one of those states, you could reflect to see exactly which state you're in. So far this has mostly been done using Reinforcement Learning, but catching it and doing it inference seems like it could be interesting to explore. And much more approachable for open source, only the big ML labs can do this sort of RL.
- TZubiri 2y agoRight. The uncertainty will be high when responding to garbage inputs and it will be distributed along many tokens. If probability(sum(tokens[:5])) < 0.5: Respond("I'm sorry I don't quite understand what you mean.")
- melenaboija 2y agoAs anthropomorphic as calling hallucinations to inaccuracies of the model. I feel anthropomorphism is part of the marketing strategy for LLMs
- jazzyjackson 2y agoHaving an oracle to chat with is a good product, but a bad framing for the tech. IMO all the broken expectations come from viewing the output as something that comes from "an other", a thing other than yourself with knowledge and experience, when really it's more of a mirror, reflecting your words back to you, enlarged or squeezed like funhouse mirrors (back in my day we didn't have skinny filters, we had to walk uphill to the pier and stand in front of a distorted piece of mercury glass! ;).
- MobiusHorizons 2y agoDid you live under water? How was the pier uphill;)
- cpeterso 2y agoThe inland area could be lower than the waterfront.
- jazzyjackson 2y agoSomehow I just knew a few of you'se would consider the implications of walking uphill to a pier
- botanical76 2y agoWhat other word would you suggest? I've seen "bullshitting" suggested, but this of course still implies intent, which AIs do not have in any typical sense of the word. I think we as a community have settled on hallucination as the best English word that approximately conveys the idea. I've seen folks on here making up words to describe it, as if that is any more useful to the victim here. The victim being the uninformed (w.r.t AI tech) layperson.
- codetrotter 2y ago“Confabulations” is sometimes mentioned as an alternative to “hallucinations”. It’s a better alternative than “bullshitting”, because “confabulating” does not have that kind of connotation of intent.
- atoav 2y agoLLMs give you a plausible chain of words, the word "hallucination" assumes intentionality that doesn't exist — as if the LLM had a "clear" state of mind and one where it felt a bit dizzy — but all of that does not describe what is going on.
- CooCooCaCha 2y agoHallucination does not imply intentionality, in fact the opposite.
- atoav 2y agowhich was my point.
- CooCooCaCha 2y agoYour point is misusing a word? The word “hallucination” in no way implies intentionality.
- atoav 2y ago
- stavros 2y agoA more apt word is "confabulation".
- CooCooCaCha 2y agoAren’t those different flavors of uncertainty?
- ben_w 2y agoI think that's the point?
- danielmarkbruce 2y agoNo, the comment reflects a misunderstanding of uncertainty. Uncertainty could be caused by all kinds of things (ie, there are flavors). That's different than saying "there are more dimensions than uncertainty".
- ben_w 2y agoThe mathematical use of the term is as you say. The article itself is uncertainty at the level of the next token rather than of the entire response, which is different: "Capital of Germany is" followed by "Berlin" is correct but it would have also been valid for the full answer to have been ", since reunification in 1990, Berlin; before this…" - correct at the conceptual level, uncertainty at the token level. Most of the users aren't aware of the maths and use words in more every-day manners, to the annoyance of those of us who care about the precise technical definitions. The listed types of uncertainty can and do have different uses in different cases. Especially the difference between "I don't know the answer" and "I do know absolutely that the answer is that nobody knows". As a chatbot it's also important to say "I don't understand your question" when appropriate, rather than to say "dunno" in response to e.g. "how do I flopragate my lycanthrope?"
- danielmarkbruce 2y agoRLHF (and DPO) are used and aren't doing token level scoring. The article is talking about inference. Most models people are actually using have gone through RLHF or DPO. So the uncertainty at inference includes all dimensions of uncertainty. A token choice can effectively be a branch from a conceptual perspective.
- glaugh 2y agoFwiw this feels deeply relevant to my usage of LLMs to structure data. I’d like exactly a good indicator of uncertainty for each bit of data.
- vark90 2y agoYou are right that uncertainty is a kinda loosely defined term. Usually people mean that it's a kind of proxy to the probability that the output of the model is correct in some sense. It's also true that uncertainty can be decomposed into "flavours". The simplest and most discussed decomposition is into aleatoric and epistemic kinds of uncertainty. Epistemic uncertainty (or model-based uncertainty) usually refers to the case, when poor output is a result of the model being presented with the kind of input which it never saw before, and should not be expected to handle correctly. Aleatoric uncertainty on the other hand is thought to be intrinsic to the data itself, think of the natural ambiguity of the task, or noisy labelling. People in the field of uncertainty estimation are very much concerned with developing methods of quantifying these different types of uncertainty, and different methods can be more sensitive to one or the other.