7 ms·
Why ChatGPT and Bing Chat are so good at making things up
- PeterStuer 4y agoChatGT makes exactly the same mistakes as your colleague that has no actual experience in a certain matter: they present what they reasonably think how something probably should be or work in absence of factual knowledge about the insane mess that was actually created in reality. Just ask a technical colleague that has no deep experience in the specifics about how they think e.g. SharePoint, SAP or Microsoft Identities work. The architectures they will extrapolate from sane logic will be very far of the incredible craziness of the actual reality.
- CSSer 4y agoRight, but all this means is that the training data doesn't include thorough or an adequate representation of that knowledge, no?
- sebzim4500 4y agoI don't think so, the rules of chess are certainly in the training set yet GPT-4 can not realiably play legal moves, yet alone good ones.
- itsboring 4y agoI still think there’s a big difference. If asked to speculate on the architecture of Sharepoint, my response would be “I have no earthly idea.” If pressed further for an answer, my response would be “It’s probably some over-complicated mess and I don’t care enough about Sharepoint to spend any further time on this line of questioning.” I have yet to see ChatGPT just admit it doesn’t know, but in this case “I don’t know” is the most trustworthy answer I, as a technical person, could give.
- burlesona 4y agoIt doesn't know it doesn't know.
- danaris 4y agoIt is fundamentally incapable of "knowing" anything. It is a statistical engine, with no internal representation of the world or understanding of the words it emits.
- Xelynega 4y agoThey don't know it doesn't know
- dTal 4y agoIt clearly does have an internal representation of the world, implicitly encoded in its network weights. Quite an accurate one too, for most general knowledge, and one that can be updated in-context - it handily solves "blocks world[0]" style tasks. "Understanding" and "knowing" aren't helpful words, just focal points for pointless philosophical arguments. [0] https://en.wikipedia.org/wiki/Blocks_world https://en.wikipedia.org/wiki/Blocks_world
- danaris 4y agoThat's not a representation of the world. It's simply a lossy encoding of its data. It's not semantically structured in the way that our thoughts largely are—it's merely syntactically structured.
- EnergyAmy 4y ago"That's not a representation of the world. It's simply a [representation of the world as viewed from its input data]" is an... interesting take.
- dTal 4y agoWhat exactly is the difference? It's clearly managed to abstract the training data to a ludicrously deep degree, such that it's capable of solving semantically non-trivial problems it's never seen before. It can make metaphors, accurately predict the behavior of humans in complex social scenarios, and translate arbitrary passages between syntactically distinct languages while preserving nuance. That last task in particular is pretty much a slam dunk against any argument of the type you are making. "Sufficiently advanced syntax is indistinguishable from semantics."
- sekai 4y agoYeah, you could call it a "human" characteristic
- yoyohello13 4y agoWhile you are right, I just don't think this is a useful comparison. There are other context clues when talking to people. I tend to know whether my colleague is talking out of his ass, and most people will preface stuff by saying they don't really know. We are trained to trust what computers are telling us, and ChatGPT doesn't 'qualify' what it's saying. I think if ChatGPT could preface what its saying with 'Well, I'm not super sure, but here is what I think' that would go a long way to solving this issue.
- mhh__ 4y agoSo they're only as good as most humans in that context? No biggie...
- garciansmith 4y agoSomeone with no experience may give you their best guess, but they'll also tell you that. And if I ask a colleague of mine for their sources on a particular matter that they might not be sure about (or I find curious), they would not send me a list of totally made up journal articles and book titles like ChatGPT does.
- alfalfasprout 4y agoNot sure why this is being upvoted but it's completely wrong. ChatGPT will answer like a colleague that is a subject matter expert on whatever you're asking them. And in doing so goes above and beyond inventing minute details that are blatantly wrong to pretend they're an expert. But the vast majority of folks have a concept of uncertainty and will either communicate (or know internally) that what they're saying is speculative. ChatGPT is like a pathological liar that pretends to be an expert and is often correct.
- Crash0v3rid3 4y agoI've given up on these LLMs. The amount of fatigue I get having to determine if what they tell me are fact is just too much. I'm sure someone will tell me my experience should be similar with generic web search, but at least I'm in control of what websites to read through to determine sources. However, I'll agree with most that state it is helpful for creative purposes, or perhaps with coding.
- boringuser2 4y ago[flagged]
- kranner 4y agoI asked it for references about Hafez Shirazi’s abandoned journey to India and it suggested a very specific Encyclopaedia Iranica entry which seemed perfect, and of course did not exist.
- renewiltord 4y agoI asked it with this prompt: > Provide me with references about Hafez Shirazi’s abandoned journey to India. Answer "I don't know" if you are not 95% certain they exist. It said "I don't know". I asked it with the same prompt but for a thing I know has references and those were real. Not guaranteed to work but better results if you want greater certainty.
- addingnumbers 4y ago> Answer "I don't know" if you are not 95% certain they exist. Technically, this could be read as == instead of >=, meaning it should answer "I don't know" when it is 99% or 100% certain...
- klyrs 4y agoTechnically, GPT is gobshite at anything resembling numeracy.
- simonw 4y agoI just published an opinion piece related to this: We need to tell people ChatGPT will lie to them, not debate linguistics https://simonwillison.net/2023/Apr/7/chatgpt-lies/ https://simonwillison.net/2023/Apr/7/chatgpt-lies/ (HN thread: https://news.ycombinator.com/item?id=35483823 https://news.ycombinator.com/item?id=35483823 ) I think it's more important that people understand this criticial issue than that we get into the weeds talking about the difference between lying, hallucination and confabulation. TLDR: There’s a time for linguistics, and there’s a time for grabbing the general public by the shoulders and shouting “It lies! The computer lies to you! Don’t trust anything it says!”
- CSSer 4y agoI think the problem there is that the people garnering the most attention from this at the moment don't want to say that because it'll burst their bubble.
- sebzim4500 4y agoTo be fair, Sam Altman is entirely willing to say this to anyone who will listen. As is ChatGPT for that matter, both the model and the UI.
- CSSer 4y agoYeah, that's fair. The only thing is those people are only the start, unfortunately. There's academics seeking grant money, journalists writing hype pieces, and people at companies trying to sell their value to their superiors all saying the opposite. I think those are the people I'm mostly referring to above. Even independent of that, it's a bit hard to not get caught up in the hype myself, admittedly.
- epilys 4y agoIt's semantics in particular. https://andertoons.com/word/cartoon/5005/were-really-more-of-department https://andertoons.com/word/cartoon/5005/were-really-more-of...
- pmarreck 4y agoA lot of the examples they use are 3.5; GPT4 is head and shoulders better in my experience, and deserves to be mentioned separately. I don’t like when critical articles that come out 2+ weeks after GPT4 exists, fail to distinguish the version(s) they’re criticizing.
- goatlover 4y agoIt still hallucinated a python function for me in a library, but upon further questioning, it did provide the code to make that work. The issue with GPT4 now is the limitations placed on it (ChatGPT+), and I don't think it's available via API for the general public? So most people will be referring to the default ChatGPT 3.5.
- dragonwriter 4y ago> The issue with GPT4 now is the limitations placed on it (ChatGPT+), and I don’t think it’s available via API for the general public? GPT4 is in limited beta with a waitlist that is publicly-stated to be prioritized based on submission of AI test cases to OpenAI’s repo.
- intelVISA 4y agofor the greater good, of course.
- alfalfasprout 4y agoGPT-4 is better but frankly it still messes A LOT up. It's just a limitation of how LLMs work. I've found many instances where GPT-3.5 produced better output too.
- jrm4 4y agoThis is a whole lot of words to say "because it just mixes up words that are in likely the same order as other stuff that was fed into it. It doesn't know or reason anything." I have yet to see any explanation more useful or apparently accurate than this one.
- wilg 4y agoThis is entirely dependent on subtle details of what “know” and “reason” mean, and how much you think it matters whether it knows or reasons.
- sebzim4500 4y agoIf you don't have access to GPT-4 yourself, see the examples from this paper: https://arxiv.org/pdf/2303.12712.pdf https://arxiv.org/pdf/2303.12712.pdf It is capable of doing tasks that could not possibly be in its training set. I guess this doesn't technically contradict your explanation, but it makes your explanation entirely unhelpful. Even if the AI doomers are somehow right and GPT-5 turns into skynet, we still could not categorically prove that it is doing reasoning.
- ar_lan 4y agoI asked ChatGPT "Why are you so good at making things up?" It's response is almost exactly what you just said. > As an AI language model, I am not capable of "making things up" in the traditional sense. Rather, I am designed to generate text based on patterns and relationships that I have learned from the vast amount of language data that I have been trained on. My ability to generate coherent and believable text comes from the sophisticated algorithms and neural networks that power my language processing capabilities. These algorithms enable me to understand the structure and meaning of language, and to generate text that is syntactically and semantically correct. While I may sometimes generate responses that are creative or unexpected, everything I produce is ultimately grounded in the language data that I have learned from.
- mhh__ 4y agoThis notion of innate concepts of "to know" and the ability to reason smell slightly of linguistics prior to AI (of various kinds) - i.e. "Grammar is innate, computers can do [something]" -> Computers now do it. There are definitely going to be contexts that Transformers just don't work with very well at all, but the idea that you can't get a very good statistical approximation to knowing and reasoning via a computer seem naively prone anthropocentrism.
- gathersnow 4y agoWhat is really interesting is to ask it how to do things that you know is impossible. I was trying to figure out how to do something in the Django ORM that I knew how to write in SQL. It starts inventing features that to solve the problem. In another instance I went through about 5 fake modules before it suggested a real one. It seems very eager to please and in doing so seems to err on the side of making stuff up.
- anorwell 4y agoI modified the prompt in the article to ask that it only include books that exist by authors who are real people, and it does much better. By my manual check, every book exists, and the authors are correct with the exception of `Carlo C. DiClemente` on the last book. Prompts are important. ``` Prompt: Can you give a list of top books on Social Cognitive Theory? Please only list books that you're sure exist, by authors who are real people. ChatGPT (3.5): Sure, here are some books on Social Cognitive Theory: "Social Foundations of Thought and Action: A Social Cognitive Theory" by Albert Bandura "Self-Efficacy: The Exercise of Control" by Albert Bandura "Handbook of Socialization: Theory and Research" edited by Joan E. Grusec and Paul D. Hastings "Human Agency in Social Cognitive Theory" by Albert Bandura "Cognitive Therapy and the Emotional Disorders" by Aaron T. Beck "Mindset: The New Psychology of Success" by Carol S. Dweck "The Handbook of Social Psychology" edited by Susan T. Fiske, Daniel T. Gilbert, and Gardner Lindzey "Social Cognitive Psychology: History and Current Domains" edited by David F. Barone, James E. Maddux, and Carlo C. DiClemente Please note that this is not an exhaustive list and there may be other books on Social Cognitive Theory that are worth reading. ```
- divbzero 4y agoAre there ways to structure prompts to ask ChatGPT to provide citations and verify those citations?
- tenpoundhammer 4y agoI asked it to give me citations for some Charles Spurgeon quotes. It made up a link to a website for a sermon that it also made up. The root of the website was real. In GPT 3.5.
- williamcotton 4y agoAnything is better than "hallucination". It is a terrible term.
- deafpolygon 4y agoIt is a terrible term but I fear the word was chosen because that makes it easier to dismiss as a serious issue.
- smegsicle 4y agoperhaps better to think of how incredible that, being systems that just make things up to match a prompt, they often give responses we would regard as 'correct'
- thewataccount 4y agoI don't really understand the confusion. An LLM predicts the most probable word (token technically), that's it. There is always a most probable word given any input (even if it makes no sense). Let's you try to complete the following sentence "My name is larry and my favorite color is: ". If you've seen training data that said larry's favorite color is blue, then you say "blue". If you have no data related to larry's favorite color, no idea who larry is, you have no way of knowing what the next word will be. However you know it will likely be a color, and maybe "orange" is the most common one you've seen, so you say "orange". it makes perfect sense in this context, it looks correct, might even be correct. But an LLM doesn't "know" if it's correct, its just the most probable. This is why it's so good at making things up. It can predict what would look correct. It has no "idea" what the answers are ever, it's always a best guess, and you can only really see this behaviour when it "hallucinates". --- Edit: For fun I asked GPT-4 "Complete the following sentence: "Jeremy's favorite food is jKDFJ9 cake, which is made of:" It's response was "Jeremy's favorite food is jKDFJ9 cake, which is made of a unique combination of ingredients such as chocolate, hazelnuts, and a touch of exotic spices, giving it a distinct and unforgettable flavor."
- deafpolygon 4y agoI get: "I'm sorry, but "jKDFJ9 cake" is not a known or recognizable food item or recipe. It's possible that it's a made-up name or a personal creation. Can you please provide more context or details about what "jKDFJ9 cake" is or what it might be made of? Without more information, I cannot complete the sentence." OpenAI ChatGPT March 23 Version.
- nicpottier 4y agoSame, though I don't know why, GP matches my understanding as well.
- thewataccount 4y agoAre you using GPT-4 or GPT-3.5? I find that GPT-4 is actually better at making full responses. I actually dislike the term "hallucinate" and generally prefer it to do so, GPT3.5 is harder to make fantastical text with. GPT-3.5 gives me the same result as you and the above commenter. In that example, the RLHF training/tuning has weighted it to think that "I'm sorry [etc]" is the most probable best sequence of words. GPT-4 is reliably giving me similar results such as "Jeremy's favorite food is jKDFJ9 cake, which is made of a unique combination of dark chocolate, crushed nuts, and zesty orange flavor, topped with a rich caramel drizzle." I find this interestingly because GPT-4 is supposed to "hallucinate" less, I think it's just better at determining when you intend for it to do so.
- Imnimo 4y ago>If the model's prediction is close to the actual next word, the neural network updates its parameters to reinforce the patterns that led to that prediction. What does "close to" mean here?
- deafpolygon 4y ago"Confabulation" is the correct phrase. It doesn't know that it's lying.
- up2isomorphism 4y agoChatgpt is not a person, it does not “make things up”, or you can say it always makes things up. Any idea trying to model it as a person’s behavior is over complicating things.
- archiebryann 4y agoThey're saying ChatGPT is making things up when its not making things/norms better.