12 ms·
We gotta stop ignoring AI's hallucination problem
- singpolyma3 3y agolol that headline. AFAICT we talk about nothing else. We're not ignoring it.
- omnicognate 3y agoI see fewer discussions of it. I've mostly given up getting involved in any. There's nothing much new to say, and people's views rarely change. I'm firmly in the camp that considers calling it "hallucination" or "getting things wrong" category errors that wrongly imply it gets anything right, and have seen nothing that remotely inclines me to revise that opinion. I recognise it as an opinion, though. It can't be proven until we have an understanding of what "understanding" is that is sufficiently concrete to be able to demonstrate that LLMs do not possess it. Likewise the view that human minds are the same kind of things as LLMs cannot be disproven until we have a sufficient understanding of how human minds work (which we most certainly do not), however obviously wrong it may seem to me. Meanwhile, as that discussion fizzles out, the commercial development of these products continues apace, so we're going to find out empirically what a world filled with them will mean, whether I think that's wise or not.
- crabmusket 3y ago> Likewise the view that human minds are the same kind of things as LLMs cannot be disproven until we have a sufficient understanding of how human minds work This seems pretty backwards to me. Why should this speculative view need to be disproven rather than proven? Sure, LLMs do some things kind of like some things human minds can do. But if you put that on a Venn diagram, the overlap would be miniscule. There's also the plain observation that LLMs are made of silicon and human minds are made of neurons- this from this you might reasonably start with the assumption that they are in fact extremely different, and the counterclaim is the one needing evidence!
- omnicognate 3y agoI'm with you in feeling that it's highly obvious that a human mind is a different sort of thing entirely to an LLM (and in more than just the trivial sense that it's implemented in wet stuff rather than dry), but plenty of people respond to "it's just a statistical model" with "well so are you", so the opposite view seems equally obvious to them. All I'm referring to is what the standard of "proof" is and that shouldn't be different for different sides of the debate. If we can therefore agree that it is not currently possible for either side to irrefutably prove the other side is wrong, then the discussion needs to be of a rather different nature if it is to have any likelihood of changing anybody's mind.
- crabmusket 3y agoI don't think there needs to be a false equivalence here. It's easy to prove that an LLM is a stastical model, since we know how they are implemented, don't we? Where's the equivalent proof that a human is a stastical model? I guess where this goes is that we don't know for sure that intelligence, or even sentience, can't emerge from a statistical model like an LLM. Which I think is a fair statement. But you can't work backwards from there to say humans and LLMs are similar!
- omnicognate 3y agoI think you're underestimating how radical (I would say nihilisic) the viewpoint that has become common among those who claim LLMs are AGI or at least show "sparks" of it is. Many of them claim that there isn't anything to emerge that hasn't already emerged. Their claim is often that there is no "intelligence" or "sentience" or "understanding" or "consciousness" distinct from the behaviours already displayed by existing statistical models. They claim that these things are illusions, that many of the philosophical questions relating to them are simply ill-posed and that the only differences between existing computational systems and human minds are ones of degree and of implementation specifics. It is this view that I am acknowledging cannot currently be disproved just as one cannot currently disprove the idea that these things are real, distinct phenomena that statistical models do not manifest. Again, I personally fall very firmly on the latter side of this debate. I'm just acknowledging what can and cannot currently be proved, and there is a genuine symmetry there. This debate is not new, and the existence of LLMs does not settle it one way or another. Edit: And re burden of proof - this isn't a court case. It's about what you can hope for in a discussion with someone you disagree with. If you can't absolutely prove someone is wrong then it's pointless to try and do so. You need to accept that you are trying to persuade, not prove/disprove. If you are both arguing positions that are currently unfalsifiable you both need to accept that or the debate will go nowhere. Or, if you think you have a proof one way or another, present that and its validity can be debated. And if so, you need to be honest about what constitutes proof and cannot reasonably be disputed. Neither "my LLM emitted this series of responses to this series of prompts" nor "I can directly perceive qualia and therefore know that consciousness is real" counts.
- pixl97 3y agoThe particular problem with 'proven' is a whole lot of things we use day to day are not proven and at best may only have statistical likelihood, but never be provable. Things like understanding, consciousness, intelligence are all things on gradients that are going to be on that scale.
- mrtranscendence 3y agoWhat does it mean to get something right? If I ask GPT-4o (or any model, really) whether it's usually hot in Topeka during the summer, it will give me the correct answer. The model may or may not understand what it means for Topeka to be hot during the summer, but it has generated text with a single reasonable interpretation which happens to include a factually correct statement. I'm comfortable saying that GPT-4o "got it right" in that case regardless of what we believe about its understanding.
- glenstein 3y agoI don't think anyone concerned about hallucinations is talking about broad brush statements which can be true generally speaking. It's about much more literal facts like code calling non-existent functions or citing non-existent court cases. Or, in my case yesterday, telling me a particular Nextcloud app could edit audio meta-data, when it in fact cannot. There are grey areas about what it means to be right to be sure, but there are also black and white areas, and it's the latter where the hallucinations are happening.
- bluGill 3y agoWho is this "we". While I see it here, I'm not seeing it talked about as much in other areas.
- jerkstate 3y agoJournalists gotta stop ignoring that experts in industry are working day and night to improve deficiencies in new world-changing technology.
- oddevan 3y agoThe article doesn't ignore that. The point is that we keep talking about that future as if it's now, and it's simply not.
- Loughla 3y agoLiterally no company who is using "ai" in the education space (my field) talks about hallucination or errors. In fact, many sell their product as hallucination free.
- duxup 3y agoThat title is a hallucination.
- explorigin 3y agoI guess there's not much training data out there to introduce ideas about facts that are not in the training data. We have to teach models that unknowns exist.
- latexr 3y ago> We have to teach models that unknowns exist. Unknowns alone aren’t the problem. See the camera example in the article. We know which steps to take to unjam film from a camera, the given answer is straight up bad.
- jerf 3y agoThe problem is, LLMs do not have a place to put "unknowns". You can train them to say "I don't know", but that doesn't come from a place of any sort of internal representation of the fact that they don't know. They don't know what they don't know, because they can't. There's nowhere for that information to be in their internal representations. And that's because they are exactly what they say they are on the tin. They are Language Models. Why would we expect a Language Model to "know" things? And to especially know what they do not know? It's modelling Language, not Knowledge, and that is not just English word play, that is a true representation of the situation. They are called "Large Language Models" for good and real reasons. The problem isn't on the LLM side. They're amazing technology. They just aren't amazing in the way we're trying to use them. We need some sort of AI that is knowledge based, that uses LLMs to input and output text but is something other than an LLM. Human brains aren't just big LLMs either. I, of course, have no idea what that is. Identifying problems is much, much easier than identifying solutions. Honestly, neural nets in general are structurally disinclined to ever say "I don't know" just by the way they work. Only a small percentage of humans ever really learn how to say "I don't know" in any significant context to any degree; we have a pretty big problem with that ourselves! LLMs are not "AI"; that is, they are not all of AI, the only thing in AI, or the the final conclusion of AI. They're just a step. And as I've said before, I suspect that once we have the next step in AI, what we're going to find is that the correct amount of resources to dedicate to the LLM portion is actually a great deal smaller. LLMs are probably super hypertrophied for what they actually should be doing and I expect 2040 to look back at our era of thinking that LLMs alone could be "AI" in much the same way we look at the idea that everyone in the world would have a Segway or something.
- guardian5x 3y agoWell, we can't even get biases out of humans in any reliable way. Maybe we will do better with machines.
- catach 3y agoWe won't, because the machines are trained on data created by humans. We won't, because the machine results people want will be those that appeal to their biases.
- HKH2 3y ago> We won't, because the machine results people want will be those that appeal to their biases. When you see an idea pushed/accelerated to an absurd conclusion, you might more easily see what's wrong with it.
- latexr 3y agoExactly. And it’s not even a new phenomenon, we know our algorithms are biased. https://www.nature.com/articles/d41586-019-03228-6 https://www.nature.com/articles/d41586-019-03228-6 https://www.technologyreview.com/2020/07/17/1005396/predictive-policing-algorithms-racist-dismantled-machine-learning-bias-criminal-justice/ https://www.technologyreview.com/2020/07/17/1005396/predicti...
- MVissers 3y agoI believe GPT-4 pre-rlhf was much more accurate overall. If you finetune it in formats that humans likes, it actually gains similar biases. I believe the ‘sparks of AGI’ talks about this, where models can much more accurately predict the probability of events than humans. After RLHF, it mimicks human bias. So it might be that we can create these models, but we just don’t like them/like to use them.
- JonChesterfield 3y agoThe idea seems to be it'll stop getting things wrong when we feed it enough information / give Nvidia enough money so we don't need to worry about that.
- websitescenes 3y agoThere is a huge backlash coming when the general public learns AI is plagued with errors and hallucinations. Companies are out there straight up selling snake oil to them right now.
- add-sub-mul-div 3y agoComing? I think the general public has already come to consider "AI" synonymous with hallucination, awkward writing, and cringe art.
- bayindirh 3y agoNo. From my experience, many people think that AI is an infallible assistant, and even some are saying that we should replace any and all tools with LLMs, and be done with it.
- kylebenzle 3y agoYou can probably safely ignore those people's opinions moving forward.
- bayindirh 3y agoInstead I keep this list updated: https://notes.bayindirh.io/notes/Lists/Discussions+about+Artificial+Intelligence https://notes.bayindirh.io/notes/Lists/Discussions+about+Art...
- mannykannot 3y agoNot once some of them incorporate hallucinating AI into important products and services.
- pyrale 3y agoThe art part is actually pretty nice, because everyone can see directly if the generated art fits their taste, and back-and-forth with the bot to get what you want is actually pretty funny. It gets frustrating sometimes, but overall it's decent as a creative activity, and because people don't expect art to be knowledge.
- vbezhenar 3y agoAI lack responsibility. I’d hallucinate to my higher ups all day if not for the fear of firing.
- lapcat 3y agoAs a test, I asked Copilot to do something impossible in JavaScript/DOM. Instead of telling me that it can't be done, Copilot provided me with source code, which of course didn't work. The reality of AI is the opposite of the horror stories. Unlike HAL 9000, AI will happily agree to open the pod bay door, even if there is no pod bay door. The AI itself is completely detached from reality, producing its own misinformation.
- weatherlite 3y agoCan you recall what it was?
- deleted 3y ago[deleted]
- nraynaud 3y agoI'm a bit confused, everybody talks about hallucinations around me, I don't feel like it's ignored.
- LtWorf 3y agoGo on r/learnprogramming or r/programming and you will find plenty of people that claim that developing without AI assistant is no longer a thing, and it's boomers who insist on doing it the old way. Of course I suspect they are 20 years old at most who never took part in a real project.
- anon22981 3y agoIt’s pretty jarring to see quite often remarks that developing with ”Gen AI” is a must and a game changer and whatnot. Usually when I have a SW problem gpt4 can’t help me. Sometimes it can speed up something marginally, but almost never no remarkable gains and sometimes it’s a net negative trying to find help from it without ever succeeding. As you said, I guess it depends on what kind of work needs to be done - I’m sure the 20 yo students can solve their OOP homework well with LLMs.
- GranularRecipe 3y agoIt really unnerves me, that we call this AI errors "hallucinations". Hallucination is an error in perception (false positive stimuli). LLM cannot hallucinate because they have no sensory organs (other than training data). I think the closest psychological conditions are confabulation (memory error) and delusion (false beliefs).
- SubiculumCode 3y agoAs a memory researcher, I agree.
- jakubmazanec 3y agoI agree with you, but I'm afraid the ship has sailed... People think "hallucinations are things that don't actually exist" and "this stuff LLM says doesn't exist, therefore it hallucinates".
- HarHarVeryFunny 3y agoThere really is no good psychological equivalent. About the best analogy I can come with is freestyle rapping where the genre requires that you just keep talking with minimal time to think. An LLM isn't an intelligent being deciding to talk when it has something to say - it's an LLM - a text continuation machine - that will generate a word every time the generation scaffolding pushes the button. Just like the freestyle rapper, it doesn't have the option of not saying something just because it has nothing to say. It's "concerned" about flow (just keeping the sentence going), not content.
- crabmusket 3y agoI love the idea of saying "the computer was just freestyling that bit".
- pixl97 3y agoIf you seperate your brain into different components and functions, which portion of your brains work like your description? If we take something like an embodied AI that has locomotion, audio/visual IO, and the ability to perform actions and part of that action loop is triggering an LLM when does it start looking more like us?
- cs702 3y agoPlease. No one is ignoring this problem. In fact, a tremendous amount of money is being invested right now to reduce/control hallucinations. It's an active area of research. Everyone with money is trying to solve the problem!
- simonw 3y agoThe Google I/O keynotes yesterday most definitely ignored the problem - it wasn't acknowledged once, and many of the demos they showed are a lot less impressive when you consider the hallucination risk.
- marcosdumay 3y ago> In fact, a tremendous amount of money is being invested right now to reduce/control hallucinations. Ok, I can only laugh on this. Those "hallucinations" are the one main feature neural networks. It's imposed by their fundamental structure, and what gives it all if its usefulness... when the people making use of it knows what they are doing. The way to reduce/control the hallucinations of a neural network is to unplug it from the power outlet.
- navjack27 3y agoAbsolutely agree. I use LLMs for their hallucinations and barely for their recall of "facts".
- cs702 3y agoThe hallucinations are a consequence of sampling from a probability distribution over all possible tokens at each step. There are a lot of very smart people trying to figure out how to sample for generative purposes while "grounding" the model so it hallucinates less. It's an active area of research.
- chickenfish 3y agoWell, does humans also have similar memory hallucination?
- gus_massa 3y agoYep. For example https://www.livemint.com/Politics/c3M3xstHMkuHMzQ5uw9OFI/False-memories-from-misinformation.html https://www.livemint.com/Politics/c3M3xstHMkuHMzQ5uw9OFI/Fal... or https://en.wikipedia.org/wiki/False_memory https://en.wikipedia.org/wiki/False_memory
- llamaimperative 3y agoWhich is why we don’t generally put human memory at critical junctures in critical systems.
- gus_massa 3y agoI agree. Moreover, as a sibling comment says, eyewitness testimony is very unreliable.
- crabmusket 3y agoI think this is a really interesting equivalence to consider. I think the difference is highlighted when someone is able to answer a question with "I don't know" or "I can't tell you that". Interestingly, LLMs can be trained to answer that way in some circumstances. But if you're cunning enough, you can trick them into forgetting they don't know something. While a human may hallucinate memories, we crucially have the ability to know when we're relying on a memory. And to think of ways we could verify our memory, or acknowledge when that's not possible.
- pixl97 3y ago>when someone is able to answer a question with "I don't know" or "I can't tell you that". Maybe LLMs are narcissists, because there are people that have problems with it, though we'd consider them to have a disorder. >we crucially have the ability to know when we're relying on a memory. When it comes to eyewitness testimony, I'd counter that we aren't nearly as good at that as we give ourselves credit for. Remembering a memory changes the memory in our wetware. In fact, I would say most of human development took eons until we started writing stuff down or documenting in a manner so we had a hard record of what the 'truth' was which then eventually turned into the scientific process of repeatability and verification.
- Jimmc414 3y agoWhen I hear Gary Marcus or others talking about this, my first thought is either they don’t understand how LLMs work, which in Gary’s case, I doubt, or they are being disingenuous. Every answer an LLM provides is a hallucination since it is just a probabilistic engine and doesn’t have an understanding of what is true or false.
- Bluecobra 3y ago> It can’t consistently figure out the simplest tasks, and yet, it’s being foisted upon us with the expectation that we celebrate the incredible mediocrity of the services these AIs provide. This is exactly how I feel about AI in it's current state. Maybe I just don't get it, but it just seems like a novelty to me right now. Like when Wolfram Alpha came out and I played around with it a few times. Copilot did help me write a one-liner with awk a few months ago so that was cool I guess.
- llamaimperative 3y agoIt is definitely oversold, but more importantly IMO it needs to be productized. We’re seeing very sloppy, ad hoc productizations but there are definitely lots of valuable, interesting problems that LLMs will help with. Probably most of them buried in the innards of archaic domains and institutions, which the public writ large will never directly interact with.
- AshamedCaptain 3y agoI think a more significant problem is the rampant copyright violations. There was huge pressure on Google when they were showing snippets that were overly large, and/or answering queries in info boxes by itself instead of redirecting users to the source. I kinda remember even John Oliver had a segment about this. And now (from the screenshots) they are just trying to show all information in an info box, not even bothering to mention which website it came from, much less link to it. It's going to rain hell.
- surajrmal 3y agoGoogle does link to the websites which contribute to the summary they provide.
- LtWorf 3y agoI asked chatgpt to tell me about the author of weborf (my bachelor thesis). It told me what it was correctly, and then claimed that some french guy had written it. The french guy does exist on github.
- nbzso 3y agoWhy? I enjoy the chaos. It will be fun when someone from corporate headquarters optimize by removing middle management and knowledge workers. After all, what is the economy? Do you need meritocracy and capitalism? If you ask Sam, the most important thing is world control trough apparatchiks centralized government, CDBC, social rating, surveillance and obedience. This is the end goal. Clearly, the quality and hallucinations of LLM's cannot be an obstacle towards the big goal of the corporate bosses. You think, this is something new? Think again. Some of us have suffered under communistic utopia. We know how it ends. It starts with big slogans: always for the people. It ends with: You will own nothing and you will be happy. Check the history, kids.:)
- intended 3y agoThere is so much pressure to deliver “GenAI” numbers/success stories. Stuff conceived with evaluations as fundamental part of their design, become unrecognizable once it’s 2 steps outside of your ability to influence. Bring up evaluation metrics and people go from enthusiastic discussions on RAG implementations, to that uncomfortable discussion where no one seems to have a common language or priors. That said, the true injury is when someone sees the prototype and eventually asks “what are your usage numbers.” Edit: Forgot this other pattern, the awkward shuffling as people reframe product features. Highly correlated with actual evaluation of their output quality.
- 2OEH8eoCRo0 3y agoI'm a total noob in this area of research but is trying to jam all human knowledge into these models the correct approach? Would a lightweight multimodal model integrated with a knowledge engine like Wolfram alpha or databases of source material be a better approach? I'm optimistic. The amount of attention this area is getting is insane.
- duxup 3y agoI don't expect perfection, far from it, and I find AI very useful. That's not unlike the internet, my coworkers, lots of things and I still work with all of them. Side note: People talk about AI's mistakes all the time. That article title is a hallucination.
- MisterDizzy 3y agoI agree that AI gets a lot wrong, and it doesn't seem to be too great at reasoning either. Potentially more dangerous is how it repeats the lies we tell ourselves. Bits of cognitive dissonance may be something that we can ignore enough to live with, but what happens when learning algos have their tentacles in a lot of different systems & have the potential for wide-reaching consequences, and they're being taught the same lies?
- kraussvonespy 3y agoI'm by no means an expert but I've seen some of what's being used to train AI, and I have to wonder if the hallucinations aren't coming from collisions of errors that got introduced while training. For example, if you don't have accurate text descriptions of the photos you want to ingest, you use BLIP or something similar to describe the photos, knowing that BLIP was also automatically trained with a certain percentage of inaccurate descriptions. So you're using a flawed AI image description model to give flawed descriptions of the photos you're ingesting, which then introduces even more errors into the photo models. The flawed photo models generate flawed pictures with flawed descriptions, which are then ingested into the next round of models. At what point do all of the accumulated errors in training make it so that all models are doing is hallucinating based on all of the flawed training data? Do we need to hire a bunch of humans to accurately classify the first couple of levels of what the models are ingesting or something similar, and would anyone actually pay for that to happen?
- tkgally 3y agoI was upset and disappointed in LLMs’ hallucination problem at first, but now I have come to terms with it. I no longer rely on them as I would a reference book or a search engine. Instead, I use them primarily for tasks that do not have right or wrong answers. One is as a translation assistant. While a translation of an individual word or phrase might be right or wrong, any longer text can be translated in many different ways, none of which is correct or incorrect, just better or worse. I do translation professionally, and now when I am translating a text I first explain its origin and purpose to several LLMs—as of yesterday, ChatGPT-4o, Claude 3 Opus, and Gemini 1.5 Pro—and have each translate it for me. I borrow from those three translations for my first draft, which I then spend plenty of time polishing and comparing with the source text. Overall, my translation time is about a third shorter when I use LLMs. The quality of the final product is also somewhat improved, as the LLMs sometimes suggest better translations than I would come up with on my own. Another useful—though sometimes frightening—application is brainstorming. I'm now teaching a discussion-based class to a group of 25 undergraduates. Last weekend, I had to come up with ideas for discussion topics for the next class session based on the reading text (a book I edited) and the homework assignments the students have submitted so far. I started doing what I normally do, which is to read those assignments and try to think up new topics that the students would find interesting and useful to discuss in class. I was going to spend most of the weekend on that. But then I remembered the huge context window of Gemini 1.5 Pro. So I uploaded all of the information about the class—the textbook, my notes and the Zoom transcripts for each class session, and all of the papers that the students have submitted so far—and I asked it to generate 50 new discussion topics as well as 20 ideas for the next homework assignment. In a couple of minutes, Gemini was able to produce the requested topics and ideas. A handful were off the mark or showed signs of hallucinations, but overall they were excellent—better and much more varied than what I could have thought up by myself. It really cut to the core of my identity as a teacher. I ended up not using any of those ideas this time, but I did share them with the students to show them what is now possible with LLMs. They seemed a bit stunned, too.
- ryandvm 3y agoCan we please stop calling it "hallucination"? That is such a poor word choice it's not even remotely accurate. Hallucinations are perceptual experiences not grounded in reality. LLMs don't have perception and they most likely don't have experiences. What LLMs are doing is bullshitting (or confabulation if you want to mind your language). Oh and by the way, humans do it all the time.
- davidguetta 3y agoLol as if human are perfect