8 ms·
ChatGPT is hallucinating fake links to its news partners' biggest investigations
- curtis3389 2y agoIt drives me nuts how OpenAI has misleadingly marketed their products, and the tech community at large has failed to clarify what their products really are. This article is so unsurprising to the point of being boring. Of course an LLM generates broken links. Why wouldn't it?
- burnte 2y agoIf you asked the LLM it would say "those look like URLs to me!" And it'd be right!
- FourierEnvy 2y agoWell as Wolfram has stated, we need to mix LLMs with some deterministic reasoning. Real knowledge based output that can be routed to for these purposes. The hard part is determining when.
- surfingdino 2y agoWe need to go back to using the AI tech stack as a tool for (broadly speaking) automated pattern recognition, i.e. it can be great at image analysis and spotting early cancerous changes in lungs, but it ought to stop at pre-qualification stage and leave it up to the human to decide what the change is and what to do with it. What the AI crowd today want is all the photos in the world to be used to train models to recognise cancerous changes and then accept blindly the following diagnosis: "it is a cancerous change of human lung tissue, therefore the patient is a labrador with chickenpox and we recommend immediate removal of the patients left wing." When you tell them it's pure garbage, they come back with answers worthy of the slimiest orators in history and solutions that Rube Goldberg would not have thought of.
- airstrike 2y agoDoesn't look like anything to me
- nerdjon 2y agoI think OpenAI and LLM's in general are a victim of their user friendliness. Honestly, when was the last time that we got a new technology that was as user friendly as talking to an LLM for the average user? For most users, its magical. It doesn't take marketing for them to start thinking this thing is "reasoning" or "thinking" after just spending a few minutes talking to it. So personally I think the problem is less the marketing (not to downplay that), and how the industry has gravited towards it. And the lack of proper education on what is not. We have to properly educate on what it is not actually doing while at the same time it sure appears to be doing exactly that since it is so good at faking it. This even exists within tech circles, we see some of it here.
- Jensson 2y ago> For most users, its magical. It doesn't take marketing for them to start thinking this thing is "reasoning" or "thinking" after just spending a few minutes talking to it. Even many smart technical people here on HN fell for that.
- andybak 2y agoIt's even stranger than that. I can go from full awareness of the nature and limitations of LLMs one minute, the treating it like an almost sentient oracle the next. It's constant struggle to remain aware of what it really is under the covers because we're just so used to engaging with language creation entities as if they had minds
- saalweachter 2y agoLLMs are fantastic tools for creating demos that are just absolutely magical, a glimpse of the future. It's only when you need to iterate on quality and go from "amazing" to "correct" that you start seeing how limited they are.
- andybak 2y agoThat's a false dichotomy. There's a huge range of applications that fall between "just a demo" and "must be reliably correct". They are just harder to productize for a broad market.
- poikroequ 2y agoAn LLM can be trained to pick a URL from a database of known URLs, or from search results. It may choose the wrong URL, of course, but at least it wouldn't hallucinate fake URLs.
- darby_nine 2y agoLLMs aren't "trained to pick a url from a database of known urls". Or at least, we have not collectively seen this yet, perhaps I am wrong. We explicitly wire things so we can use LLMs to generate data useful to index with existing search databases.... acting like LLMs are responsible for the reliability of identifying the source document is disingenuous when a large part of the value of an LLM as an asset is its role laundering other peoples' intellectual property without attribution.
- poikroequ 2y agoTry out phind.com for an example. The LLM generates a query for the search engine to fetch web results. It can include clickable links to the sources in its response. They accomplish this by training/fine-tuning the LLM to output special tokens, or special syntax, which is interpreted in code to perform some action, such as calling out to an API. This is how chatgpt and Gemini are able to automatically search the web and generate images and such. Yes, LLMs just output tokens, but those tokens can be interpreted to perform actions in code.
- SirMaster 2y agoWhy can't it just check the URL to see if it's valid and actually contains the data it thinks it contains before recommending it to the user. Don't modern LLMs like GPT4 have access to load web pages? The downside is it will slow down the output I guess.
- consumer451 2y agoI am currently more worried about the clear and present threat of weaponized bullshit "AI" than I am about the theoretical threat of AGI. One would imagine that the FTC could require warning labels of "AI" products: "This product will confidently lie and cannot be trusted for factual information," etc. This warning would be especially useful for decision makers in commercial and government domains.
- foobiekr 2y agoThe tech industry has long, long, long been a rapacious pump and dump scheme where the tech is barely relevant and most of the industry has been harvesting violations of social norms and regulations (tracking and ad tech, Uber, etc.). The only reason any of us defend it is that's how we make money. Honestly a lot of people in tech should be a little ashamed.
- nineteen999 2y agoIt is this kind of stuff that makes me feel very warm and fuzzy that I work in a tech sector that literally helps saves lives (emergency services). I've worked in tech in banking, gambling, software development, internet service provision, and even the postal service. They were all extremely scummy compared to the sector I work in now.
- reaperducer 2y agoHealthcare tech worker here. I know exactly how you feel. "What did I do today? Helped a bunch of doctors get the information they needed to diagnose someone's obscure disease. What did you do?" "Oh, I worked on ways to make my company's app more addictive to 12-year-olds, and look less like gambling." I'm happy to be in the first group.
- digging 2y ago> Honestly a lot of people in tech should be a little ashamed. I think it's true that a lot of people are a little ashamed already, but that a lot of people in tech should be more actively opposed to what's happening. As a small example, part of the paycheck I get from placing hidden F_cebook tracking on my corporate website to analyze the behaviors of people who could not possibly consent, I donate to the EFF. Actually it's quite difficult to be politically engaged against abusive tech in the same way I may or may not be engaged in local politics, but money is one thing we all have.
- 3pt14159 2y agoI am not ashamed of pumping OpenAI in general and ChatGPT in particular. The sheer amount of technological heft they've brought in such a short time is breathtaking. I've literally had an hour plus conversation, by voice, with ChatGPT and I was engaged throughout the entire affair. The tech isn't perfect when it comes to some things like links and what have you, but what does work is short of a miracle.
- whateveracct 2y agoReminds me of how facial recognition software was hyped and sold to police departments.
- fnordpiglet 2y agoAs you read the entire article you will note that OpenAI said they’ve not yet released the feature. The press release was about inking the deal not releasing the feature. They are still implementing the feature. This is mostly an article about how unreleased features in software that are in development still don’t yet work in production and proving that the unreleased features are still unreleased, then dressing it up as a scandal.
- dontlikeyoueith 2y ago> the tech community at large has failed to clarify what their products really are Most of the tech community is as clueless about ML as the general public.
- deleted 2y ago[deleted]
- LetsGetTechnicl 2y agoLLM's?? Hallucinating? Color me shocked!!
- reaperducer 2y agoWe really have to get over the term "hallucinating." It's "lying." Just like "misinformation" is also lying. Call it what it is. I know the point of "hallucinating" is to distinguish it from a falsehood told with intent. But when a drunkard tells a lie in a bar, it's still a lie, even if the intent to deceive isn't there.
- redeux 2y agoClearly those companies should use ChatGPT to generate those URLs. That way when ChatGPT guesses what the URL should be, it’ll be correct! /s in case it’s not obvious
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- gs17 2y agoReally they just need to make the 404 page run the requested URL through a RAG setup and redirect to the most likely article if one exists. And if the article doesn't exist, it can simply make it up!
- deleted 2y ago[deleted]
- throwaway4aday 2y agoIt could be done and it might actually be useful. You could fine tune one of OpenAI's models on the material owned by the company and use RAG to create bespoke summaries or quotations to fulfill the user's request but if you're going to do that it might be better for each of those companies to provide an API that ChatGPT (or any other LLM) can call directly and provide a few capabilities like getting all pages published within some date range or by a specific author or in such and such category and also provides a search endpoint that can handle both full text and similarity search. Then as long as ChatGPT correctly makes the function call when asked for a link it should be able to return the exact URL or a handful of results in cases where it is unclear which article the user is asking for e.g. if they misremembered the title and author and provided a vague recollection of the content.
- netruk44 2y agoI wonder how difficult it would be to make a website that, instead of serving 404's, a model does a semantic search using the nonexistent URL to come up with a URL that actually does exist and most closely matches the "intent" of the invalid one.
- kalleboo 2y agoHow about just using an LLM to generate a page that matches what the URL appears to try to link to? (not being serious here)
- throwaway4aday 2y agoThat's called Websim my friend and it's a heck of a lot of fun https://websim.ai https://websim.ai This is what it generated for one of the non-existent articles https://websim.ai/c/P32MdBI15Ytxwwlth https://websim.ai/c/P32MdBI15Ytxwwlth
- gwbas1c 2y agoNot too hard, if the site already has some kind of a search engine, it can feed the URL to the search engine. The safe thing to do is return a 404 with a few suggested links. The problem is that "guessing" the intent of the URL will have unintended consequences. A URL will no longer be guaranteed to point to what you think it points to. This fundamentally breaks the assumption of what a URL is. How could this happen? 1: Site creates article foo.com/chocolate_bars 2: Someone accidentally links to foo.com/cocoa_bars, which serves foo.com/chocolate_bars 3: Site creates article foo.com/coco_bars 4: Now foo.com/cocoa_bars serves foo.com/coco_bars
- AlexandrB 2y agoYes, this sounds like a nightmare. Also pity poor archive.org if they try to archive a web site set up like this - every URL will resolve to a valid page.
- netruk44 2y agoMy thought was to serve a 307 (or some other redirect HTTP code, clearly I am not a web developer) instead of a 404, not to serve the found content “as” the url that didn’t exist. But agreed this would not be good general practice. It was mostly a thought experiment.
- barbariangrunge 2y agoI think the bigger issue is trust. The chat it is no longer trying to return objective information. “OpenAI” is letting companies pay to have it promote their content—is that accurate?
- deleted 2y ago[deleted]
- AlexandrB 2y agoTo paraphrase Casablanca: "I'm shocked, shocked to find that paid promotions are going on in here!" Objectivity is hard to monetize, promoting random crap is not.
- NovemberWhiskey 2y agoLLMs don't return objective information. Never have. You're getting a statistically-probable token stream based on the training corpus and the prompt.
- ToValueFunfetti 2y agoI don't think that follows. If ChatGPT tells me that two masses attract one another via gravity, that's objective information regardless of whether it uses a statistical process to produce it out of training data or contacts an oracle to directly query the divine will of the gods. LLMs don't always return objective information, but they typically do.
- chrisjj 2y ago> ChatGPT is hallucinating fake links There's no "hallicinating" and no "faking". A computer program is generating faulty links, period.
- RHSeeger 2y agoThe English language evolves over time. In the context of AI, the word "hallucinating" is used to mean "presenting false information in a way that clearly indicates it _thinks_ the information is true". [1] _thinks_ isn't the right word here, but that's what it looks like when reading the responses
- chrisjj 2y agoGiven _think_ is the wrong word, so is _hallucinate_. Hallucinate us being used primarily to promote the idea these programs think. "presenting false information in a way that clearly indicates it [not] _thinks_ the information is true" is just PR spin on "program output is incorrect".
- mostlysimilar 2y agoThe English language evolves over time. In the context of AI, the word "hallucinating" has been popularized by the corporations creating these products because "it's fucking wrong" doesn't have quite the same ring to it for marketing and general bullshitting purposes.
- localfirst 2y agoWe seem to be exiting the hype phase
- sixhobbits 2y agoI like a bit of ChatGPT bashing, same as the next guy, but this seems a bit unfair. The partnerships have been signed, but as far as I know OpenAI has made no indication that the implementation is done or even started yet, so this story is basically "ChatGPT hallucinates URLs", which is a pretty well-known issue. The 'of partner websites' bit seems a bit shoe horned in to try prove some point. It hallucinates URLs for any site, and I don't think the partnerships are relevant here.
- ToValueFunfetti 2y ago>OpenAI has made no indication that the implementation is done or even started yet In fact, they explicitly told this journalist that they haven't implemented it: >OpenAI told me in a statement that it has not yet launched the citation features promised in its licensing contracts.
- Kiro 2y agoI very seldom experience hallucinations and I'm a really heavy user of ChatGPT. Can someone give me a prompt that is likely to generate hallucinations?
- LeonardoTolstoy 2y agoFactual info specifically looking for a link of something obscure where lots of similar (but incorrect) links exist. E.g. "what is the IMDb link for the film Ghost Fever?" Throwing that in Chatbot Arena will usually result in a random link back or the model will say it can't do it. GPT-4o gave a random link, Claude said it couldn't do it, oddly geneini-advanced-0514 got it right, so hard to tell if they are using Google in the background or if they do better. But yeah usually IMDb links come back right for big films but tend to be hallucinated for smaller ones. That's just an example.
- NovemberWhiskey 2y agoAsk it to provide references. I just asked it for an explanation of the Fresnel effect in VHF communications and to provide references. The first was Wikipedia. One of the other links was a site about radar that had no relevant content. One was to this: https://popups.uliege.be/accueil/ https://popups.uliege.be/accueil/ ... and one was to this: https://vividcomm.com https://vividcomm.com
- meroes 2y agoI’m a lowly human contractor who does a kind of reinforcement learning, with inducing hallucinations as one of my main goals. I can’t give any work away and I’m not at my desk today, but having the AI rely on its own reasoning is one of my heuristics for tripping it up. For example, give it something it has to break down into a series of steps, as that’s making it rely on its own “logical reasoning”, and gives it many places to mess up. Don’t let it just depend on some external structure. Make it commit to its own bootstrapping capabilities. I liked ones about can you draw certain letters (capital, English, etc) without lifting your pen off the page or setting up a physical pattern, like starting the alphabet on a chess board and having it give you the letter at a certain tile of the pattern continues. It may also depend on the model but giving a mathematical sequence an asking for the next term is also a common failure point. I’m limited by what I can talk about and I don’t want to test across more models than I have to, but suffice to say if the goal is hallucination, it becomes apparent. If you ask it to walk you through each step of its reasoning it will be more likely to hallucinate somewhere in there as well. The chessboard one might sound like it can just rely on the structure of chessboards and the English alphabet, but you’re also forcing it to understand some pattern, which is harder for it. Like, initialize the pattern with 3 letters not adjacent to each other so it has to “think” about the pattern rather than just repeat an easily identifiable one.
- Edmond 2y agoThis seems a result of relying on the LLM to accurately extract information that needs to be exact. I touch on this from my own experience: https://youtu.be/cs5cbxDClbM?si=IQIFAD38cVzLCs55&t=486 https://youtu.be/cs5cbxDClbM?si=IQIFAD38cVzLCs55&t=486 Basically if you have the actual "factual" information, use it directly instead of hoping the LLM will accurately extract it and use it as part of a function call. In this case they already know what the accurate URLs are, just use it.
- dartos 2y agoThe LLM _might_ do what you want. Where I currently work, our function calls regularly fail only to succeed flawlessly on a retry. (I believe we’re on the order of 10s of millions openai calls a day) These are non-deterministic systems. I wouldn’t even trust them to accurately extract text until you did a beam search or something similar to kind of average out different LLM outputs
- klyrs 2y agoIt's a special kind of hell, taking a hallucinogen and finding yourself in a white buttondown and slacks, writing ad copy.
- deleted 2y ago[deleted]
- maxbaines 2y agoIt's easy to think of the models in this context, rather than the application built on top of the model, in this case ChatGPT. ChatGPT is not simply the gpt-4o model, it's the model, a system prompt, tools and a virtual environment running python. I built my own app, when using it i don't get links to partners because i don't mention such a thing in the system prompt.
- 1vuio0pswjnm7 2y agoPerhaps ChatGPT's inherent flaws will drive www users to use newspaper websites instead. This could be a win for journalism, which seems to be the adversary (not the partner^1) of Silicon Valley. 1. Except for all the "tech"-columnists pumping out marketing gibberish Assuming the lawyers for the newspapers were smart enough to retain the right to serve the news without "AI", i.e., they did not agree to funnel users to ChatGPT via their own websites, they become the only authoritative sources, what with www search engine indexes no longer publicly searchable, queries for specific resources having been replaced with "prompts" to word soup generators. With the absurd limits on number or search results and now this "AI" nonsense, I've been preparing to transition away from the popular www search engines toward searching only websites of selected publishers, i.e., authoritative sources. Cut out the middleman. I do all searching from the command line and create mixed SERPs over time, similar to the "metasearch" concept but using site-specific search for non-commercial content instead of www search engines. It has been working well for me.