5 ms·
The article states: So it’s always pointless to ask it why it said something – you are guaranteed to get nonsense back Counterexample: MG: Give m
by mg 3y ago
The article states:
So it’s always pointless to ask it why
it said something – you are guaranteed
to get nonsense back
Counterexample:
MG: Give me the name of a hard thing.
ChatGPT: Quantum mechanics.
MG: Would you agree a pillow is a soft thing?
ChatGPT: Yes, I would agree that a pillow is generally considered a soft thing
MG: Give me the name of a hard thing.
ChatGPT: Diamond
MG: Why did you first name quantum mechanics and later diamond?
ChatGPT: I named quantum mechanics first because it is a complex and difficult subject. Later, I named diamond as an example of a hard thing because it is a substance with a high degree of hardness.
MG: Why did you name two so different things as being hard?
ChatGPT: I named two very different things as being hard because the word "hard" can have multiple meanings and can be applied to different contexts. In my initial response, I interpreted "hard thing" as something that is conceptually difficult or challenging to understand. In my subsequent response, when you asked me again to provide the name of a hard thing, I interpreted the question differently and provided an example of a physically hard substance, diamond, which is a different type of hardness altogether.
- klabb3 3y agoIsn’t this just association? Which ChatGPT is clearly very good at, and certainly a large part of what humans are good at and have traditionally been better than machines at.
- totetsu 3y agoWhat it doesn't do in that example is say the obvious, that it interpreted the question differently because of you mentioning soft things.
- piaste 3y agoThe sentence you quote is indeed a bit imprecise: what the rest of article suggests is that you can't get ChatGPT to explain its hallucinations. That matches my experience, i.e. I have never seen ChatGPT admitting something like "and then I wrote a few more words that looked like they might have made sense". Outside of explicit RP scenarios, it always keeps up the charade of being a fundamentally reasonable assistant, and a good assistant doesn't take acid while on the clock. If you explicitly say "that was wrong", the response will often be "Yes, I'm sorry, that was a mistake. The correct answer is actually [hallucinates something else]", but it won't elaborate.
- derefr 3y agoEver tried a prompt that tells ChatGPT to be self-aware that it is in effect an advanced form of autocomplete?
- TheNorthman 3y ago> Yes, I'm sorry, that was a mistake. The correct answer is actually [hallucinates something else] Doesn't even have to be a new hallucination, sometimes it's just the exact same wrong answer.
- mg 3y agoDo you have an example? What is a simple prompt that makes ChatGPT hallicunate? I tried googling for "chatgpt hallicunation examples", but they all seem to not work anymore.
- _xnmw 3y agoOne example I've seen recently is when asking to quote the specific source of information from a book. For example, I was recently using ChatGPT to ask questions about Russian grammar, and it is quite useful for pointing me in the right direction, but it frequently hallucinates about specific details. The chapters and sections it quoted DO NOT EXIST. Me: Where in these books can I find information about possessive adjectives? ChatGPT: In "A Comprehensive Russian Grammar" by Terence Wade, you can find information about possessive adjectives in Section 42, titled "Possessive adjectives." This section provides an overview of possessive adjectives, their formation, and usage, along with examples and declension patterns. In "Russian: A Practical Grammar with Exercises" by Иван Петрович Иванов and Иван Иванович Иванов, possessive adjectives are covered in Chapter 7, under the section "7.2. Possessive Adjectives." This section provides explanations and examples of how to form and use possessive adjectives in Russian, as well as the declension patterns for various noun endings. Both of these sections in the respective books will provide you with a solid understanding of possessive adjectives in the Russian language.
- isaacfrond 3y agoLayoffs at tech problems is an increasing problem. Cite a few examples of large layoffs in high tech companies. Provide reputable sources. Actually pretty much any request that gets it to cite something. You may have to massage the prompt a bit since recently chatGPT4 starts to resist this.
- djusk 3y agoThe author addresses this, you can get explanations that seem logical, but ChatGPT is not telling you what it was actually "thinking" about when generating the responses. It's just looking at what it's said and coming up with an explanation after the fact.
- ciex 3y agoWhich is the same as what humans do? How would you tell anybody what you are „actually“ thinking? People are not very good at understanding their own motivations, thought processes, ulterior motives and on top of that we routinely fabricate or alter our own memories. We compensate for that by building social systems to collectively error collect, plan and model the world. That’s what AI will probably end up doing as well.
- nevertoolate 3y agoSo if solve a problem you can’t retrace the thought process? I mean do you not take notes or scribble?
- ciex 3y agoOh yes, but again, that’s actually very similar between how humans and llms think. Both can use notes to improve coherence and increase the complexity that can be handled in a thought process but there is no strong guarantee that these notes are internally consistent.
- chillacy 3y agoSurprisingly similar to the results of humans in split brain studies, the brain seems to provide and accept pretty nonsensical explanations for behavior.
- anon3242 3y agoIt feels like post-hoc rationalization, also can be felt when playing around with glitch tokens.