10 ms·
Responses from LLMs are not facts
- xd1936 1y agoA simple static webpage, inspired by motherfuckingwebsite.com, comicsanscriminal.com, etc.
- purplecats 1y agoironically, would be more valuable if this cited each of its claims rather than just "further reading". "But Stopcitingai Said "
- deleted 1y ago[deleted]
- fao_ 1y agoMentally rename the "further reading" to "citations", then?
- purplecats 1y agoeach claim should be individually cited, not left to the user to investigate where in all the further reading each claim corresponds to
- deleted 1y ago[deleted]
- mr3martinis 1y agoBosses love it when you call them foolish.
- dude250711 1y agoSweet of you to think LLM consumers read things.
- foxfired 1y agoWe used to say "stop copying and pasting from stackoverflow without reading it first". Everything changed, yet everything is the same.
- alecsm 1y ago[flagged]
- hekkle 1y agoOMG YES, that site needed to die! I posted a few times on subjects I was an expert in, and hence they were difficult issues, and no one would ever answer them. The few other times I posted they were questions about things I wasn't an expert in, hence why I was asking, and my god, it was like I was pulling them away from their busy schedules and costing them time at work. It's like you don't have to answer if you have something better to do.
- ceejayoz 1y agoChatGPT would be worthless without training material like SO.
- CamperBob2 1y agoNot really. LLMs are good at indexing and digesting documentation, up to and including actual source code, and answering questions about it. And they never "Vote to close as duplicate" because somebody asked something vaguely similar 10 years ago about a completely different platform and didn't get a good answer even then. Stack Overflow is the taxi industry to AI's Uber. We needed it at one point, but it really always sucked, and unsurprisingly some people took exception to that and built something better, or at least different.
- PaulDavisThe1st 1y ago> LLMs are good at indexing and digesting documentation, up to and including actual source code, and answering questions about it. Requires citations not in evidence. Source code and documentation rarely co-exist, and even the best source code is not even close to well-described by documentation of the software it is a part of. I basically call BS.
- themafia 1y agoIf your boss says this to you, quit.
- echelon 1y agoLLMs are still better than the trash-filled waste bin Google Search has become.
- IncreasePosts 1y agoIt's the same content
- stavros 1y ago> Responses from Large Language Models like ChatGPT, Claude, or Gemini are not facts. > They’re predicting what words are most likely to come next in a sequence. I wish we'd move away from these reductive statements that sound like they mean something but are actually a non-sequitur. "Articles on Wikipedia are not facts. They're variations in magnetic flux on a platter transferred over the network". Yeah, that doesn't make them not facts, though. The LLM should simply cite its sources, and so should Wikipedia, a human, or a dog, otherwise I'm not believing any of them. Especially the human.
- dullcrisp 1y agoArticles on Wikipedia are not facts. They’re the product of community contributions on a topic. I don’t think that’s really a non-sequitur, but I guess it depends on what’s meant by facts in your epistemology.
- glitchc 1y agoWe can call wikipedia content facts by consensus. It's hard to say the same for LLMs since the input is not curated for accuracy, even though the wikipedia content is a subset of the entire training corpus. In short, the curation is the key differentiator between the two.
- drdaeman 1y ago> not curated for accuracy I thought accuracy is one of metrics that the models are trained for…
- charcircuit 1y agoConsensus of what the news media says. Wikipedia doesn't actually care if the content is true.
- amiga386 1y agohttps://en.wikipedia.org/wiki/Wikipedia:Verifiability https://en.wikipedia.org/wiki/Wikipedia:Verifiability > Even if you are sure something is true, it must have been previously published in a reliable source before you can add it. If reliable sources disagree with each other, then maintain a neutral point of view and present what the various sources say, giving each side its due weight. Wikipedia cares that its contents are taken from reliable sources, which can be independently verified. Not all news media are reliable sources, and in fact academic papers and journals and published books are generally more reliable than news media.
- rlayton2 1y agoEven in small companies, its important to discuss what the expectations around AI are. In the absence of any further requirements (i.e. assuming privacy is not a major issue, regulatory issues etc), it can be as simple as clearly saying: "You can use AI, but you are ultimately responsible for what you deliver. It is expected you verify the data, test the code, and otherwise validate the responses." Something as simple as that gives an expectation, without being overbearing to start with.
- class4behavior 1y agoAnyone's basic expectation is that your sourcedis a valid one, but like Wikipedia is not a source and neither are LLMs/MMMs.
- SilverElfin 1y agoThis snarky site may make you feel smart but really there’s no reason to cite and trust anything, and AI isn’t much worse than alternatives. Even peer review isn’t the guarantee you think it is. AI is often right as well and we should keep that in mind.
- exasperaited 1y agoAI is never right. It’s also never wrong. LLMs bullshit us, in the truest sense: there’s no distinction between right and wrong, no investment in being correct, no remorse or embarrassment whatsoever when wrong. They don’t really deserve to be called “right” when they spit out words that happen to be. They aren’t “wrong” when they spit out words that happen to be. They don’t care so we shouldn’t project these higher notions onto them. It’s worthless empty air either way. Prose with the value bargained down to the average of all prose.
- delis-thumbs-7e 1y agoFirst of all, you can only verify the informations correctness if you know fairly much about the topic. Did you know that Sweden lost the battle of Pultava, because syfilis was affecting Charles XII’s brain? If you don’t believe me, I’m pretty sure I can gaslight some model or another to agree with me. That you cannot do with a peer-reviewed journal and even less so with a respected book on the subject. Better LLM or even internet forums are more useful you know about the subject. You can use them for sparring, testing theories and just for fun, but you shpuld not use them to learn about a subject. For that you need a book and some practice, maybe a lecture or two won’t hurt. Of course there is nuance to this, but in general they just are not trustworthy and will most likely never be.
- Aeolun 1y agoI think my main response to these messages is: “If ChatGPT is more trustworthy than me, the expert you hired, what do you have me for?” I can spend hours refuting and explaining why what ChatGPT told you doesn’t apply in our situation, or you can… just trust me?
- paulcole 1y agoThe people who get the most haughty about insisting that they be “just trusted” tend to be the people who are the least trustworthy.
- Aeolun 1y agoI don’t think I’m haughty when I don’t want to try and convey 20 years of experience in a few sentences every time some quotes chatgpt?
- paulcole 1y agoYou don't think ChatGPT is ever correct enough to just go with it? You really want to prove your experience every single time?
- drewbug01 1y agoIf someone is taking the time to refute ChatGPT’s output and telling you why the answer isn’t applicable in a given situation, it certainly implies that ChatGPT wasn’t “correct enough” at all. What situations do you think it’s fine to be “correct enough?”
- paulcole 1y agoBut this person says they want to refute it in every situation. Some people seem to make a hobby of refuting the output of others. So no, I don’t trust the implication that if somebody spends time refuting it that it must be worth refuting. In my experience (with both people-output and ChatGPT-output) my goal is to not refute anything unless it absolutely positively must be refuted. If it’s a low-stakes situation where another person has an idea that seems like it might/will probably work, let them go nuts and give it a shot. I’ll offer quick feedback or guiding questions but I have 0 interest in refuting it even if I think there’s a chance it’ll go wrong. They can learn by doing.
- gus_massa 1y ago> Imagine someone who has read thousands of books, but doesn’t remember where they read what. That sound like me! Well I probably read only a hundred, but I also mostly forgot the sources. I can halucinate a source, like "there is (probably) a Schaum book about that".
- delis-thumbs-7e 1y agoYou mean I should not use ”But gus_massa on HN said that!” as a final authority on every subject anymore?
- gus_massa 1y agoYou still can cite me as a final authority on every subject [1]. But when I'm in doubt, I try to make that clear. Also, in many cases I link to Wikipedia or somewhere I consider serious. (tokenadult used to add many good references for his posts, it's a pity he is not coming so often here.) [1] Don't worry. I'm always right. See [1].
- parpfish 1y agobut the thing is... this is just how learning works in general. As you learn facts, the source evaporates away as the content is integrated with the rest of your knowledge. once upon a time somebody told you (or you read) "Paris is the capital of France". For some period, you'd have a distinct episodic memory of the time and place for how you learned this fact. But as time passes, that episodic memory fades and eventually you're just left with the content "Paris is the capital of France" without the original source.
- rezonant 1y agoYes but if someone asks me for a source that Paris is the capital of France, I will go find a legitimate source for that information to include in my response, not hallucinate a source and steadfastly claim it is real. AIs not remembering the source for the original fact is not the problem, the problem is that asking them to make sure their claims are supported by sources does not preclude them from making up the sources themselves.
- bgwalter 1y agoAlternatively, give the same prompt to another model and get a completely different answer. Sometimes the opposite. Or give the same prompt to the same model after its latest fine tuning and get a completely different answer. Or warm up the model with leading prompts and get a different answer. These things are just addictive toys, nothing more.
- amlib 1y agoYou can make the exact same question to the same LLM and the "artificial entropy" they inject into the inference process will be enough to make up a completely different response.
- 9x39 1y ago"Can you share the chat so we can look at it together?" Asking for the receipts so you can figure out where they put their thumb on the scale is more illuminating.
- notatoad 1y agoyou're more polite than me, but that's essentially the same response as what i have to people citing chatGPT. i just say "ChatGPT told me that's wrong". if somebody thinks that unverified LLM output is relevant to a conversation, i don't want to have to defend why it shouldn't be part of the conversion, i want to put the responsibility for justifying it back onto them.
- deleted 1y ago[deleted]
- exasperaited 1y agoThen they start texting it to you in Apple Messages and expecting you to treat it with as much weight as their own words.
- nerder92 1y agoIs this lmgtfy of the AI era?
- dcre 1y agoThis is more like Let Me Not Google That For You and You Shouldn’t Either
- exasperaited 1y agoNo, don’t do this. It’s as bad as the “no hello” thing. If it bothers you when people do the “chatgpt said” thing (and it should), put your concerns into your own words. Or at least respond with an article in the news that you can discuss with that person. Responding with one of these sites is just as worthless and devoid of interpersonal investment as responding with AI. Don’t be that person.
- siliconpotato 1y agothese sites are useful to elaborate on your policy/culture decisions rather than sending a direct response to the NPC from the other team who always does the hello thing e.g. writing a wiki page with etiquette on how to talk to your team? and a link to the nohello site as a footnote to your guidance
- exasperaited 1y agoI know that the US work culture is passive-aggressive by default now but this is the worst of it. If you care about an issue, you get your point across with your own words in a discussion where they use their words, not with a thing that lets you hide. What is the specific impact in your business? What are your personal feelings? What are the reasons why people using AI to get their point across?
- gnarlouse 1y agoThis is so passive aggressive it’s fireable—or at the very least unpromotable.
- geocrasher 1y agoLLM's follow the old adage of "Garbage In, Garbage Out". LLM's work great for things that are well documented and understood. If you use LLM's to understand things that are poorly understood in general, you're going to get poor information because the source was poor. Garbage in, Garbage out. They are also terrible at understanding context unless you specify everything quite explicitly. In the tech support world, we get people arguing about a recommended course of action because ChatGPT said it should be something else. And it should, in the context for which the answer was originally given. But in proprietary systems that are largely undocumented (publicly) they fall apart fast.
- exasperaited 1y agoYou’re going to get poor information presented with equal certainty as good information, though. And when you ask it to correct it, more bad information with a cheery, worthless apology.
- geocrasher 1y agoThe ability to spot poor information is what keeps the end user a vital part of the process. LLM's don't think. [Most] humans do.
- deleted 1y ago[deleted]
- paulcole 1y agoI remember the guy who created Comic Sans said, “If you love Comic Sans you don’t know anything about typography and should get a new hobby. If you hate Comic Sans you also don’t know anything about typography and should get a new hobby.” I feel like this applies to AI as well.
- hamasho 1y agoRemind me this useful URL when boss says "we need carousel". https://shouldiuseacarousel.com/ https://shouldiuseacarousel.com/
- ProofHouse 1y agoTell me you know nothing about AI without telling me you know nothing about AI
- ixxie 1y agoI know its disturbing to many, but there is something nice about the post-truth moment: it feels like more people are actually questioning things more than when I grew up in the 90s/00s. I think we need to shift towards a socionormative understanding of knowledge; as Rorty put it: "a fact is just something we can't be bothered to argue about". I agree with him that talking about truth isn't so useful for moving our culture forward. We should be talking about how to negotiate the diverse vocabularies of discursive communities as they increasingly clash in our globalized culture. Dialectical exclusion is the cultural catastrophe of the day.
- yellow_postit 1y agoThis feels like getting taught in school not to cite Wikipedia when the actual digital literacy challenge is deeper— learn where the info comes from and to critically think.
- deleted 1y ago[deleted]
- taneq 1y agoExactly! It’s the credibility of the data once cross referenced with other sources that really matters. It could be a paper on arxiv or it could be a 4chan post, what matters is if it checks out.
- Spivak 1y agoWell you shouldn't cite Wikipedia in your paper for the same reason you shouldn't cite LLMs, they're tertiary sources. You shouldn't cite a paper book encyclopedia either. It has nothing to do with digital literacy so I'm sorry if that's what was taught to you. You should look to an encyclopedia for information about all manner of topics. Someone did the work of organizing, verifying, and cross-referencing the information from disparate sources for you. It doesn't mean the information is untrustworthy, if that were true the paper you wrote in class would be untrustworthy which is absurd, no?
- analog8374 1y agoWhen you mostly get your facts secondhand from utter strangers (as most of us do), any statement made clearly and confidently enough is indistinguishable from fact.
- Brendinooo 1y agoI mostly just think this is a bad response to a real problem. Attitude problems aside[0], if you lead with "Responses from Large Language Models like ChatGPT, Claude, or Gemini are not facts", most people are probably going to respond with some form of "well it said this one fact and I know for sure that one was right" and move on from this unconvinced of anything. I'm not sure what a better approach is though. Honestly, "Don’t copy-paste something that a chatbot said and send it to someone as if that’s authoritative" feels like a better starting point. Another comment in this thread about asking to share the prompt and demonstrating how it can be manipulated could help. Or talking about LLM bias. I dunno. P.S. what's up with those form inputs/submits for the good/bad uses? [0]: "lmgtfy" or even "rtfm" seemed to fade after the novelty wore off and everyone realized it was a jerk move to use terms like that. Or maybe it's a generational thing?
- ninetyninenine 1y agoWho on the face of the earth doesn’t know this? Scroll through the comments and if you find one person who is like “oh shit really?? I had no idea!” Then you’re hallucinating. This page isn’t designed to be informative it’s designed as self affirmation to people who really hate AI. Ai is not fully reliable. But it’s reliable enough to use as a tool. But there are tons of people who hate it and want to inform others it’s bad even though the world already knows. They see this and are like “yeah I’m right”
- lngnmn2 1y ago[dead]
- aspbee555 1y agoit really can't be that difficult to have a llm reference a known factual reference before giving an answer. Its really good at figuring out what you want and what to say, not far off to check references
- ssivark 1y agoThink for a minute about how undefined and vague what you are asking is. Unless the specific claim you have made is close to verbatim in some reference (in which case you're doing search & retrieval), it's incredibly hard to "automate" the interpretation of meaning from the source doc, your query, and the token being generated by the LLM -- such that you can give a sensible citation. There's a long tail of cases where this will break, and the moment you expose this functionality to users they will have the easy cases for breakfast, quickly start exploring the long tail and complain about failures.
- mcv 1y agoChatGPT told me I function two pay scales higher than I'm being paid. I think that's going to be my response when someone cites ChatGPT as an authority to me.
- sanskarix 1y ago[dead]
- tim333 1y agoThey can be facts. Sometimes they get thing right, sometimes wrong like other sources. Now "Responses from LLMs are not facts" in not a fact. I guess it provides a kind of meta illustration of not believing everything you read. A more subtle problem is they are poor as citations as they can change rapidly unlike saying say, last weeks NYT said so and so.
- ffsm8 1y agoI personally like to call it "asking the oracle" https://en.wikipedia.org/wiki/Oracle https://en.wikipedia.org/wiki/Oracle Because it makes it clear that this might as well be entirely made up BS, just like the oracle's were frequently doing
- alexey-salmin 1y agoI usually reply with this image: https://www.reddit.com/r/MemeTemplatesOfficial/comments/lhwy5h/i_guess_ill_just_go_fuck_myself_then/ https://www.reddit.com/r/MemeTemplatesOfficial/comments/lhwy...
- codyswann 1y agoNo. Facts are facts. Opinions are opinions. And statements of fact are unverified facts. I wish people would start understanding the difference. "Ice cream is cold" is an opinion. "Ice cream melts at 50 degrees Fahrenheit" is a statement of fact.
- GaryBluto 1y agoWhile I somewhat agree with the sentiment, I find these sorts of sites obnoxiously lecturous and won't be read by the target audience anyway, it's just too snarky.
- Rileyen 1y agoJust because AI sounds convincing does not mean it is telling the truth. I have believed something it said before because it sounded right, only to find out later that it was completely wrong. These days I never rely only on what AI says and always check the original source. The most reliable way I have found to use it is to treat it like a smarter search engine.
- mmis1000 1y agoSometimes I wonder, why people ask AIs to validate their output by performing tool search. But not do the search or even open the reference AI provided by themselves. In any case, whether it is real person or AI do the task for you. It's the user who submitted the result to be responsible for it. Why do people think it's bad to do this with real human and suddenly be OK to do this with an AI?
- lionkor 11mo agoIt's quite easy to just go to the source quoted by the LLM, read it or skim it, and quote that.