12 ms·

LLMs Will Always Hallucinate, and We Need to Live with This

• jmakov 2y ago
  Maybe time for getting synth minds from guessing to reason.
• zyklonix 2y ago
  We might as well embrace them: https://github.com/DivergentAI/dreamGPT https://github.com/DivergentAI/dreamGPT
• bicx 2y ago
  I treat LLMs like a fallible being, the same way I treat humans. I don’t just trust output implicitly, and I accept help with tasks knowing I am taking a certain degree of risk. Mostly, my experience has been very positive with GPT-4o / ChatGPT and GitHub copilot with that in mind. I use each constantly throughout the day.
  • ulbu 2y ago
    I treat them as always hallucinating and it just so happens that, by accident, they sometimes produce results that resemble intentional, considered or otherwise veritable to the human observer. The accident is sometimes of a high probability, but still an accident. Humans are similar, but we are the standard for what’s to be considered a hallucination, clinically speaking. For us, a hallucination is a cause of concern. For llms, it’s just one of all the possible results. Monkeys bashing on a typewriter. This difference is an essential one, imo.
  • everdrive 2y ago
    One big difference is that at least some people have a healthy sense for when they may be wrong. This sort of meta-cognitive introspection is currently not possible for an LLM. For instance, let's say I asked someone "do you know the first 10 elements of the periodic table of elements?" Most people would be able to accurately say "honestly I'm not sure what comes after Helium." But an LLM will just make up some bullshit, at least some of the time.
    • TheGeminon 2y ago
      There are ways to gauge the confidence of the LLM (token probabilities over the response, generating multiple outputs and checking consistency), but yeah that’s outside the LLM itself. You could feed the info back to the LLM as a status/message I suppose
      • christianqchung 2y ago
        The problem is also that the model may have a very high confidence in token probability and is still wrong, but I'm sure it could help in some cases.
      • Der_Einzige 2y ago
        The idea of hooking LLMs back up to themselves, i.e. giving them token prob information somehow or even giving them control over the settings they use to prompt themselves is AWESOME and I cannot believe that no one has seriously done this yet. I've done it in some jupyter notebooks and the results are really neat, especially since LLMs can be made with a tiny bit of extra code to generate a context "timer" that they wait before they prompt themselves to respond, creating a proper conversational agent system (i.e. not the walkie talkie systems of today) I wrote a paper that mentioned doing things like this for having LLMs act as AI art directors: https://arxiv.org/abs/2311.03716 https://arxiv.org/abs/2311.03716
    • The_Colonel 2y ago
      That's indeed the biggest problem, because it limits its usefulness to questions for which you can verify the correctness. (Don't get me wrong, you can still get a lot of utility out of that, since for many problems finding the [candidate] solutions is much more difficult than verifying them) OT, but this also reminds me how much I despise bullshitters. Sometimes right, something wrong, but always confident. In the end, you can't trust what they say.
  • giantrobot 2y ago
    > I treat LLMs like a fallible being, the same way I treat humans. The issue is LLMs are not marketed in this way. They're marketed as all knowing oracles to people that have been conditioned to just accept the first result Google gives them.
• mxwsn 2y ago
  OK - there's always a nonzero chance of hallucination. There's also a non-zero chance that macroscale objects can do quantum tunnelling, but no one is arguing that we "need to live with this" fact. A theoretical proof of the impossibility of reaching 0% probability of some event is nice, but in practice it says little about whether we can exponentially decrease the probability of it happening or not to effectively mitigate risk.
  • panarky 2y ago
    Exactly. LLMs will sometimes be inaccurate. So are humans. When LLMs are clearly better than humans for specific use cases, we don't need 100% perfection. Autonomous cars will sometimes cause accidents. So do humans. When AVs are clearly safer than humans for specific driving scenarios, we don't need 100% perfection.
    • talldayo 2y ago
      > When AVs are clearly safer than humans for specific driving scenarios, we don't need 100% perfection. People didn't stop refining the calculator once it was fast enough to beat a human. It's reasonable to expect absolute idempotent perfection from a robot designed to manufacture text.
      • wkat4242 2y ago
        Yeah if the computer had been wrong 3 out of 10 times it never would have been a thing.
      • kfarr 2y ago
        Not sure absolute perfection is a concept that can exist in the universe of words.
      • tourmalinetaco 2y ago
        Maybe, down the line. The calculator went through a long period of perfecting until it became as powerful as they are today. It’s only natural LLMs will also take time. And much like calculators moving from stepped drums, to vacuum tubes, to finally transistors, the way we build LLMs are sure to change. Although I’m not quite sure idempotence is something LLMs are capable of.
• simonw 2y ago
  A key skill necessary to work effectively with LLMs is learning how to use technology that is fundamentally unreliable and non-deterministic. A lot of people appear to find this hurdle almost impossible to overcome.
  • jampekka 2y ago
    LLMs are not fundamentally non-deterministic. It's trivial to do e.g. greedy sampling generation.
  • threeseed 2y ago
    Honesty and accuracy builds trust. And when you trust something it reduces the cognitive load because you don't have to build a mental model of the different ways it could be deceiving you and how to handle it. Which is why for me at least when I use LLMs I find them useful but stressful.
    • BeetleB 2y ago
      Everything you said applies to regular search engines too. There's always a cognitive load in using them. Often the load is less with an LLM. Other times it's not. Can't use either of these tools for all problems. Pick the right tool for the right problem.
• jampekka 2y ago
  I'm of the opinion that the current architectures are fundamentally ridden with "hallucinations" that will severely limit their practical usage (including very much what the hype thinks they could do). But this article puts an impossible limit to what it is to "not-hallucinate". It essentially restates well known fundamental limitations of formal systems and mechanistic computation and then presents the trivial result that LLMs also share these limitations. Unless some dualism or speculative supercomputational quantum stuff is invoked, this holds very much to humans too.
  • diggan 2y ago
    > fundamentally ridden with "hallucinations" that will severely limit their practical usage On the other hand, a LLM that got rid of "hallucinations" is basically just a thing that copy-paste at that point. The interesting properties from LLMs comes from the fact that it can kind of make things up but still make them believable.
    • jampekka 2y ago
      As per this article, even copy-paste hallucinate e.g. because there are no infinite datasets.
    • SoftTalker 2y ago
      > can kind of make things up but still make them believable This is the definition of a bullshitter, by the way.
      • Kerb_ 2y ago
        Or, you know, fiction writer. Some of us like little stories.
        • _heimdall 2y ago
          Its important that anyone reading fiction knows its fiction. Without that we can run into a situation where a fictional story on the radio convinces the public that we are under alien attack in what could be called a war of the worlds.
• treebeard901 2y ago
  If everyone else can hallucinate along with it then problem solved.
• feverzsj 2y ago
  Maybe, it's time for the bubble to burst.
  • rvz 2y ago
    But but before that we need to achieve what we call "AGI" first. Even before that we need to define it, first and the reality is no-one knows what "AGI" even is. Thus it could be anything. The fact that Sam doesn't believe that AGI has been "achieved" yet even after GPT-3.5, ChatGPT, GPT-4 (multi-modal), and with o1 (Strawberry) suggests that what AGI really means is to capture the creation and work of billions, raise hundreds of billions of dollars and for everyone to be on their UBI based scheme, whilst they enrich themselves as the bubble continues. Seems like the hallucinations are an excuse to say that AGI has not yet been achieved. So it time to raise billions more money on training, energy costs on inference all for it to continue to hallucinate. Once all the value has been captured by OpenAI and the insiders cash out, THEN they would want the bubble to burst with 95% of AI startups disappearing (Except OpenAI).
  • aoeusnth1 2y ago
    LLMs amplify both intelligence and stupidity, which is why Terence Tao finds them about at the level of a mediocre graduate student (and getting better), and you can’t wait for them to die.
• deleted 2y ago
  [deleted]
• lolinder 2y ago
  > By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is going awry in the normal functioning of the model, which subtly suggests that if we could just identify what went awry we could get rid of the problem and restore normal cognitive function to the LLM. The trouble is that the normal functioning of the model is simply to produce plausible-sounding text. A "hallucination" is not a malfunction of the model, it's a value judgement we assign to the resulting text. All it says is that the text produced is not fit for purpose. Seen through that lens it's obvious that mitigating hallucinations and creating "alignment" are actually identical problems, and we won't solve one without the other.
  • gwervc 2y ago
    This comment should be pinned at the top of any LLM-related comment section.
    • Moru 2y ago
      It should be part of every AI related story on the news. Just like they keep saying "X, formerly Twitter".
    • elif 2y ago
      Nah it's quite pedantic to say that 'this neologism does not encapsulate the meaning it's meant to' This is the nature of language evolution. Everyone knows what hallucination means with respect to AI, without trying to confer to its definition the baggage of a term used for centuries as a human psychology term.
      • lolinder 2y ago
        Neologism undersells what this term is being used for. It's a technical term of art that's created its own semantic category in LLM research that separates "text generated that is factually inaccurate according to ${sources}" from "text generated that is morally repugnant to ${individuals}" or "text generated that ${governments} want to censor". These three categories are entirely identical at a technological level, so I think it's entirely reasonable to flag that serious LLM researchers are treating them as distinct categories of problems when they're fundamentally not at all distinct. This isn't just a case of linguistic pedantry, this is a case of the language actively impeding a proper understanding of the problem by the researchers who are working on that problem.
• advael 2y ago
  It's crazy to me that we managed to get such an exciting technology both theoretically and as a practical tool and still managed to make it into a bubbly hype wave because business people want it to be an automation technology, which is just a poor fit for what they actually do It's kind of cool that we can make mathematical arguments for this, but the idea that generative models can function as universal automation is a fiction mostly being pushed by non-technical business and finance people, and it's a good demonstration of how we've let such people drive the priorities of technological development and adoption for far too long A common argument I see folks make is that humans are fallible too. Yes, no shit. No automation even close to as fallible as a human at its task could function as an automation. When we automate, we remove human accountability and human versatility from the equation entirely, and can scale the error accumulation far beyond human capability. Thus, an automation that actually works needs drastically superhuman reliability, which is why functioning automations are usually narrow-domain machines
  • akira2501 2y ago
    > want it to be an automation technology They want it to be a wage reduction technology. Everything else you've noticed is a direct consequence of this, and only this, so the analysis doesn't actually need to be any deeper than that.
    • advael 2y ago
      Business culture's tribal knowledge about technology seems to have by and large devolved into wanting only bossware, that all gizmos should function as a means of controlling their suppliers, their labor force, or their customers. I think this is a good sign that the current economic incentives aren't functioning in a desirable way for most people and this needs significant intervention
• seydor 2y ago
  Isn't that obvious without invoking godel's theorem etc?
  • deleted 2y ago
    [deleted]
• zer00eyz 2y ago
  Shakes fist at clouds... Back in my day we called these "bugs" and if you didn't fix them your program didn't work. Jest aside, there is a long list of "flaws" in LLMS that no one seems to be addressing. Hallucinations, Cut off dates, Lack of true reasoning (the parlor tricks to get there don't cut it), size/cost constraints... LLM's face the same issues as expert systems, without the constant input of experts (subject matter) your llm becomes quickly outdated and useless, for all but the most trivial of tasks.
• OutOfHere 2y ago
  This seems to miss the point, which is how to minimize hallucinations to a desirable level. Good prompts refined over time can minimize hallucinations by a significant degree, but they cannot fully eliminate them.
• danenania 2y ago
  Perplexity does a pretty good job on this. I find myself reaching for it first when looking for a factual answer or doing research. It can still make mistakes but the hallucination rate is very low. It feels comparable to a google search in terms of accuracy. Pure LLMs are better for brainstorming or thinking through a task.
• ninetyninenine 2y ago
  Incomplete training data is kind of a pointless thing to measure. Isn’t incomplete data the whole point of learning in general? The reason why we have machine learning is because data was incomplete. If we had complete data we don’t need ml. We just build a function that maps the input to output based off the complete data. Machine learning is about filling in the gaps based off of a prediction. In fact this is what learning in general is doing. It means this whole thing about incomplete data applies to human intelligence and learning as well. Everything this theory is going after basically has application learning and intelligence in general. So sure you can say that LLMs will always hallucinate. But humans will also always hallucinate. The real problem that needs to be solved is: how do we get LLMs to hallucinate in the same way humans hallucinate?
  • skzv 2y ago
    Yes, but it also makes a huge difference whether we are asking the model to interpolate or extrapolate. Generally speaking, models perform much better on the former task, and have big problems with the latter.
    • _heimdall 2y ago
      Without any ability to reason about the known facts, are we better off with LLMs trying to interpolate at all rather than acting as a huge search space that returns only references? If an LLM has the exact answer needed it could simply be returned without needing to be rephrased or predicted at all. If the exact answer is not found, or if the LLM attempts to paraphrase the answer through prediction, isn't it already extrapolating? That doesn't even get to the point where it is attempting to combine multiple pieces of training data or fill in blanks that it hasn't seen.
  • abernard1 2y ago
    > Machine learning is about filling in the gaps based off of a prediction. I think this is a generous interpretation of network-based ML. ML was designed to solve problems. We had lots of data, and we knew large amounts of data could derive functions (networks) as opposed to deliberate construction of algorithms with GOFAI. But "intelligence" with ML as it stands now is not how humans think. Humans do not need millions of examples of cats to know what a cat is. They might need two or three, and they can permanently identify them later. Moreover, they don't need to see all sorts of "representative" cats. A human could see a single instance of black cat and identify all other types of house cats as cats correctly. (And they do: just observe children). Intelligence is the ability to come up with a solution without previous knowledge. The more intelligent an entity is, the less data it needs. As we approach more intelligent systems, they will need less data to be effective, not more.
• reilly3000 2y ago
  Better given them some dried frog pills.
• deleted 2y ago
  [deleted]
• leobg 2y ago
  Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them. Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing it, we relate that thought to other things we know. We may do “sanity checks“ internally, often habitually without even realizing it. Therefore, we should not expect an LLM to generate the correct answer immediately without giving it space for reflection. In fact, if you observe your thinking, you might notice that your thought process often takes on different roles and personas. Rarely do you answer a question from just one persona. Instead, most of your answers are the result of internal discussion and compromise. We also create additional context, such as imagining the consequences of saying the answer we have in mind. Thoughts like that are only possible once an initial “draft” answer is formed in your head. So, to evaluate the intelligence of an LLM based on its first “gut reaction” to a prompt is probably misguided. Let me know if you need any further revisions!
  • nickpsecurity 2y ago
    Our brains also seem to tie our thoughts to observed reality in some way. The parts that do sensing and reasoning interact with the parts that handle memory. Different types of memory exist to handle trade offs. Memory of what makes sense also grows in strength compared to random things we observed. The LLM’s don’t seem to be doing these things. Their design is weaker than the brain on mitigating hallucinations. For brain-inspired research, I’d look at portions of the brain that seem to be abnormal in people with hallucinations. Then, models of how they work. Then, see if we can apply that to LLM’s. My other idea was models of things like the hippocampus applied to NN’s. That’s already being done by a number of researchers, though.
  • soared 2y ago
    I also like comparing a human thought experiment like Einstein (?) would do to forcing an llm to write code to answer a question. Yes you can make a good guess, but making many smaller obvious decisions that lead to an answer is a stronger process.
• nailuj 2y ago
  To tire a comparison to human thinking, you can conceive of it as hallucinations too, we just have another layer behind the hallucinations that evaluates each one and tries to integrate them with what we believe to be true. You can observe this when you're about to fall asleep or are snoozing, sometimes you go down wild thought paths until the critical thinking part of your brain kicks in with "everything you've been thinking about these past 10 seconds is total incoherent nonsense". Dream logic. In that sense, a hallucinating system seems like a promising step towards stronger AI. AI systems simply are lacking a way to test their beliefs against a real world in the way we can, so natural laws, historical information, art and fiction exist on the same epistemological level. This is a problem when integrating them into a useful theory because there is no cost to getting the fundamentals wrong.
• rw_panic0_0 2y ago
  since it doesn't have emotions I believe
• davesque 2y ago
  The way that LLMs hallucinate now seems to have everything to do with the way in which they represent knowledge. Just look at the cost function. It's called log likelihood for a reason. The only real goal is to produce a sequence of tokens that are plausible in the most abstract sense, not consistent with concepts in a sound model of reality. Consider that when models hallucinate, they are still doing what we trained them to do quite well, which is to at least produce a text that is likely. So they implicitly fall back onto more general patterns in the training data i.e. grammar and simple word choice. I have to imagine that the right architectural changes could still completely or mostly solve the hallucination problem. But it still seems like an open question as to whether we could make those changes and still get a model that can be trained efficiently. Update: I took out the first sentence where I said "I don't agree" because I don't feel that I've given the paper a careful enough read to determine if the authors aren't in fact agreeing with me.
  • yencabulator 2y ago
    I posit that when someone figures out those architectural changes, the result won't be called an LLM anymore, and the paper will be correct.
    • davesque 2y ago
      Yep, could be.
  • noobermin 2y ago
    You can never completely solve the problem because it's mathematically undecideable, which you probably didn't need this preprint to intuit. That said, a better question is whether you can get good enough performance or not.
• nybsjytm 2y ago
  Does it matter that, like so much in the Math for AI sphere, core details seem to be totally bungled? e.g. see various comments in this thread https://x.com/waltonstevenj/status/1834327595862950207 https://x.com/waltonstevenj/status/1834327595862950207
• gdiamos 2y ago
  Disagree - https://arxiv.org/abs/2406.17642 https://arxiv.org/abs/2406.17642 We cover halting problem and intractable problems in the related work. Of course LLMs cannot give answers to intractable problems. I also don’t see why you should call an answer of “I cannot compute that” to a halting problem question a hallucination.
• lsy 2y ago
  LLM and other generative output can only be useful for a purpose or not useful. Creating a generative model that only produces absolute truths (as if this was possible, or there even were such a thing) would make them useless for creative pursuits, jokes, and many of the other purposes to which people want to put them. You can’t generate a cowboy frog emoji with a perfectly reality-faithful model. To me this means two things: 1. Generative models can only be helpful for tasks where the user can already decide whether the output is useful. Retrieving a fact the user doesn’t already know is not one of those use cases. Making memes or emojis or stories that the user finds enjoyable might be. Writing pro forma texts that the user can proofread also might be. 2. There’s probably no successful business model for LLMs or generative models that is not already possible with the current generation of models. If you haven’t figured out a business model for an LLM that is “60% accurate” on some benchmark, there won’t be anything acceptable for an LLM that is “90% accurate”, so boiling yet another ocean to get there is not the golden path to profit. Rather, it will be up to companies and startups to create features that leverage the existing models and profit that way rather than investing in compute, etc.
• renjimen 2y ago
  Models are often wrong but sometimes useful. Models that provide answers couched in a certain level of confidence are miscalibrated when all answers are given confidently. New training paradigms attempt to better calibrate model confidence in post-training, but clearly there are competing incentives to give answers confidently given the economics of the AI arms race.
• TMWNN 2y ago
  How goes the research on whether hallucinations are the AI equivalent of human imagination, or daydreaming?
• pkphilip 2y ago
  Hallucinations in LLM will severely affect its usage in scenarios where such hallucinations are completely unacceptable - and there are many such scenarios. This is a good thing because it will mean that human intelligence and oversight will continue to be needed.
• reliableturing 2y ago
  I’m not sure what this paper is supposed to prove and find it rather trivial. > All of the LLMs knowledge comes from data. Therefore,… a larger more complete dataset is a solution for hallucination. Not being able to include everything in the training data is the whole point of intelligence. This also holds for humans. If sufficiently intelligent it should be able to infer new knowledge, refuting the very first assumption at the core of the work.
• badsandwitch 2y ago
  Due to the limitations of gradient descent and training data we are limited in the architectures that are viable. All the top LLM's are decoder-only for efficiency reasons and all models train on the production of text because we are not able to train on the thoughts behind the text. Something that often gives me pause is the consideration that it is actually possible to come up with an architecture which has a good chance of being capable of being an AGI (RNNs, transformers etc as dynamical systems) but the model weights that would allow it to happen cannot be found because gradient descent will fail or not even be viable.
• mrkramer 2y ago
  So hallucinations are something like cancer, it will have sooner or later, in another words, it is inevitable.
  • mrkramer 2y ago
    it will happen sooner or later*
• willcipriano 2y ago
  When will I see AI dialogue in video games? Imagine a RPG where instead of picking from a series of pre recorded dialogues, you could just talk to that villager. If it worked it would be mind blowing. The first studio to really pull it off in the AAA game would rake in the cash. That seems like the lowest hanging fruit to me, like we would do that long before we have AI going over someone's medical records. If the major game studios aren't confident enough in the tech to have it write dialogue for a Disney character for fear of it saying the wrong thing, I'm not ready for it to anything in the real world.
  • mewse 2y ago
    As a game developer, I'm absolutely not going to risk my players' engagement with the game by putting character personalities, dialog, and (therefore) plot into the hands of an AI that's going to play out differently for each player. No way, no how. That's where all my game's value is - that's what pays my rent - and I will not be handing it off to a glorified RNG.
    • deleted 2y ago
      [deleted]
    • deleted 2y ago
      [deleted]
• fsndz 2y ago
  We can't get rid of hallucinations. Hallucinations are a feature not a bug. A recent study by researchers Jim Waldo and Soline Boussard highlights the risks associated with this limitation. In their analysis, they tested several prominent models, including ChatGPT-3.5, ChatGPT-4, Llama, and Google’s Gemini. The researchers found that while the models performed well on well-known topics with a large body of available data, they often struggled with subjects that had limited or contentious information, resulting in inconsistencies and errors. This challenge is particularly concerning in fields where accuracy is critical, such as scientific research, politics, or legal matters. For instance, the study noted that LLMs could produce inaccurate citations, misattribute quotes, or provide factually wrong information that might appear convincing but lacks a solid foundation. Such errors can lead to real-world consequences, as seen in cases where professionals have relied on LLM-generated content for tasks like legal research or coding, only to discover later that the information was incorrect. https://www.lycee.ai/blog/llm-hallucinations-report https://www.lycee.ai/blog/llm-hallucinations-report
• ndespres 2y ago
  We don’t need to “live with this”. We can just not use them, ignore them, or argue against their proliferation and acceptance, as I will continue doing.
  • CatWChainsaw 2y ago
    This is "anti-progress", and we must always pursue progress even if it leads us to a self-made reality-melting hellmouth. Onward to Wonderland, I say!
    • _heimdall 2y ago
      Gotta love "progress" as the ultimate goal. Ignore how you got here, whether you made a wrong turn, or even if you're happy as-is and don't need to change. Just keep pushing forward and moving around the deck chairs!
  • inglor_cz 2y ago
    Technically, you are right. Donald Knuth still doesn't use e-mail, after all. But for the global "we" entity, it is almost certain that it is not going to heed your call.
  • aoeusnth1 2y ago
    I want to use them. Lots of other people do, too. Feel free to get left behind.
    • _heimdall 2y ago
      You're assuming that the ending is already written and the long term value of LLMs is already known. Someone not using them could get left behind if LLMs turn out that to have a consistent multiplier effect on productivity. That person also may not actually care about being "left behind". Someone not using them could also save a bunch of time and effort if LLMs don't pan out and don't add a meaningful value over the long run. That's not to say they don't add any value, LLMs could be relegated to the role of a much better predictive text engine that gets used regularly but isn't itself a large enough gain to leave anyone behind for not using predictive text.
• rapatel0 2y ago
  Been saying this from the beginning. Let's look at comparitor of a human result. What is the likelihood that a junior college student with access to google will generate a "hallucination" after reading a textbook and doing some basic research on a given topic. Probably pretty high. In our culture, we're often told to fake it till you make it. How many of us are probabilistic-ly hallucinating knowledge we've regurgitate from other sources?
  • fny 2y ago
    If you ask a student to solve a problem while admitting when they don't an answer, they will stop at generating gook for an answer. LLMs on the other hand regularly spew bogus with high confidence.
    • rapatel0 2y ago
      You’re idealizing. Students also hallucinate / bullshit their way through things.
• throwawaymaths 2y ago
  LLMs hallucinate because probs -> tokens erase confidence values and it's difficult to assign confidences to strings of tokens, especially if you don't know where to start and stop counting (one word? one sentence?) Is there a reason to believe this is not solvable as literally an API change? The necessary data are all there.
• _cs2017_ 2y ago
  Useless trash paper. It's like saying any object can disappear and reappear anywhere in the universe due to quantum physics, so there's no point studying physics or engineering. Just maybe we care if the probability of that happening is 10%, 0.00001%, or 1e-1000%.
• jongjong 2y ago
  The hallucinations seem to be related to AI's agreeableness. They always seem to tell you what you want to hear except when it goes against significant social narratives. It's like LLMs know all possible alternative theories (including contradictory ones) and which one it brings up depends on how you phrase the question and how much you already know about the subject. The more accurate information you bring into the question, the more accurate information you get out of it. If you're not very knowledgeable, you will only be able to tap into junior level knowledge. If you ask the kinds of questions that an expert would ask, then it will answer like an expert.
• 99112000 2y ago
  Humans try to replicate the human brain in software and are surprised it sometimes spits out dumb things a human could have said.
• ramshanker 2y ago
  Few Humans (all?) will always Hallucinate, and we already live with this. ;)
• russfink 2y ago
  Sounds like a missed STTNG story line. I can imagine that such a “Data,” were we ever to build one, would hallucinate from time to time.
• m3kw9 2y ago
  They did say each token is generated using probability, not certainty, given that there is a chance it produces wrong tokens
  • aptsurdist 2y ago
    This gets at the heart of the problem. It doesn’t produce the wrong tokens. The tokens are right. It’s the data that was “wrong”. Or at least it was weighted “incorrectly” (according to the judge living outside the data with their own context they decide is true) If you feed AI conspiracy theories and then it tells you Elvis is still alive, that’s an input problem not an algorithm problem. Now, getting to an AI that doesn’t “hallucinate” is a little more complicated than simply filtering out “conspiracy theories” from data, but IMhO it’s not many orders of magnitudes away. Far from insurmountable in a couple Moores law cycles. I think human brains operate on the same principal of divining next tokens. We’re just judging AI for not saying the tokens we like best even though we feed AI garbage in and don’t tell AI what it should even care about. “AI doesn’t live here” it wasn’t born “here.” Someday soon we’ll probably give AI guard rails to respond considering the context of society (the programmers version of society), and it will probably hallucinate less than most humans.
• data_maan 2y ago
  Just from the way this paper is written (badly, all kinds of LaTeX errors), my belief that something meaningful was proved here, that some nice mathematical theory has been developed, is low. Example: The first 10 pages are meaningless bla
  • noobermin 2y ago
    Sorry but you're just wrong. There are issues but the paper is written well enough. The content (whether this is really a novel enough idea) is debateable because anyone could have told you that LLMs aren't going to develop the halting algorithm.
    • data_maan 2y ago
      Have you actually read the paper/know how a ML paper should be written? Here are some of the issues: - section 1.4.3: Can you explain how societal consequences of LLM hallucinations are in any way relevant for a paper that claims in the abstract to use mathematical theories ("computational theory", although that is an error too, they probably mean computability theory)? At best, such a section should be in the appendix, if not a separate paper. - section 1.2.2: What is with the strange subsections 1.2.2.1, 1.2.2.2 that they use for enumeration? - basic LaTeX errors, e.g. page 19, at L=, spacing is all messed up, authors confuse using "<" with "\langle", EFC. So no, I'm afraid you are wrong. The paper violates many of the unspoken rules of how a paper should be written (which can be learned by reading a lot of ML papers, which I guess the authors haven't done) and based on this alone, as it is, wouldn't make it into a mediocre conference, let alone the ICML, ICLR, NeurIPS.
• mrjin 2y ago
  LLMs can neither understand nor hallucinate. All LLMs are just picking tokens based on probability. So doesn't matter how plausible the outputs look, the reasons lead to the output are absolutely NOT what we expect them to be. But such ugly fact cannot be admitted or the party would be stopped.
  • aptsurdist 2y ago
    Human brains are also just picking tokens tho. A beautiful illusion of insight and thought in the chaos noise of information. But out of the chaos, the emergence of thought is real. It’s just not exclusive to humans. I mean… even magicians (mentalists) can reliably hack humans into generating the next token they want you to generate.
• Animats 2y ago
  Oh, not that again. Didn't we see this argument about three weeks ago. A 100% correct LLM may be impossible. A LLM checker that produces a confidence value may be possible. We sure need one. Although last week's proposal for one wasn't very good. When someone says something practical can't be done because of the halting problem, they're probably going in the wrong direction. The authors are all from something called "UnitedWeCare", which offers "AI-Powered Holistic Mental Health Solutions". Not sure what to make of that.
• cynicalpeace 2y ago
  Humans do it too. We just call it “being wrong”
• irrational 2y ago
  I prefer confabulate over hallucinate. Confabulate - To fill in gaps in one's memory with fabrications that one believes to be facts. Hallucinate - To wander; to go astray; to err; to blunder; -- used of mental processes Confabulation sounds a lot more like what LLMs actually do.
  • Kwpolska 2y ago
    "Bullshit" is even more appropriate.
• aptsurdist 2y ago
  But humans also hallucinate. And humans habitually stray from the “truth” too. It’s always seemed to me that getting AI to be more accurate isn’t a math problem, it’s getting AI to “care” about what is true - aka better defining what truth is- aka what sources should be cited with what weights. We can’t even keep humans in society from believing in the stupidest conspiracy theories. When humans get their knowledge from sources indiscriminately, they also parrot stupid shit that isn’t real. Now enter Gödel’s incompleteness Theorem: there is no perfect tie between language and reality. Super interesting. But this isn’t the issue. Or at least it’s not more of an issue for robots than it is for humans. If/when humans deliver “accurate” results in our dialogs, it’s because we’ve been trained to care about what is “accuracy” (as defined by society’s chosen sources) Remember that AI “doesn’t live here.” It’s swimming in a mess of noisy context without guidance for what it should care about. IMHO, as soon as we train AI to “care” at a basic level about what we culturally agree is “true” the hallucinations will diminish to be far smaller than the hallucinations of most humans. I’m honestly not sure if that will be a good thing or the start of something horrifying.
• namaria 2y ago
  LLMs will go the way of the 'expert systems'. We're gonna wonder why we ever thought that was gonna happen. I just recommend you don't pidgeonhole yourself and an AI professional because it's gonna be awfully cold outside pretty soon.
• 9733105200 2y ago
  [flagged]