4 ms·

Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all,

by leobg 2y ago

Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them. Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing it, we relate that thought to other things we know. We may do “sanity checks“ internally, often habitually without even realizing it. Therefore, we should not expect an LLM to generate the correct answer immediately without giving it space for reflection. In fact, if you observe your thinking, you might notice that your thought process often takes on different roles and personas. Rarely do you answer a question from just one persona. Instead, most of your answers are the result of internal discussion and compromise. We also create additional context, such as imagining the consequences of saying the answer we have in mind. Thoughts like that are only possible once an initial “draft” answer is formed in your head. So, to evaluate the intelligence of an LLM based on its first “gut reaction” to a prompt is probably misguided. Let me know if you need any further revisions!

nickpsecurity 2y ago
Our brains also seem to tie our thoughts to observed reality in some way. The parts that do sensing and reasoning interact with the parts that handle memory. Different types of memory exist to handle trade offs. Memory of what makes sense also grows in strength compared to random things we observed. The LLM’s don’t seem to be doing these things. Their design is weaker than the brain on mitigating hallucinations. For brain-inspired research, I’d look at portions of the brain that seem to be abnormal in people with hallucinations. Then, models of how they work. Then, see if we can apply that to LLM’s. My other idea was models of things like the hippocampus applied to NN’s. That’s already being done by a number of researchers, though.
soared 2y ago
I also like comparing a human thought experiment like Einstein (?) would do to forcing an llm to write code to answer a question. Yes you can make a good guess, but making many smaller obvious decisions that lead to an answer is a stronger process.
gus_massa 2y ago
> A human does not do this. You obviously had never asked me anything. (Specialy tech questions while drinking a cup of cofee.) If I had a cent for every wrong answer, I'd be already a millionair.
- NemoNobody 2y ago
  Why?? To defend AI you used yourself as an example of how we can also be that dumb too. I don't understand. Your example isn't true - what the OP posted is the human condition regarding this particular topic. You, as a human being obviously kno better than to blurt out the first thing that pop into your head - you even have different preset iterations of acceptable things to blurt in certain situations solely to avoid saying the wrong thing like - I'm sorry for your loss. Thoughts and prayers" and stuff like "Yes, Boss" or all the many rules of politeness, all of that is second nature to you, a prevents from blurting shit out. Lastly, how do dumb questions in the mornings with coffee at a tech meeting in any way compare to an AI hallucination?? Did you ever reply with information that you completely made up, has seemingly little to do with the question and doesn't appear to make any logical or reasonable sense as to why that's your answer or how you even got there?? That's clearly not the behavior of an "awake" or sentient thing. That is perhaps the simplest way for normal people to "get it" is by realizing what a hallucination is and that their toddler is likely more capable of comprehending context. You dismissed a plainly stated and correct position, with self depreciating humor - for why?
  - gus_massa 2y ago
    Sorry for the long delay... Sometime when I'm working, I see items in a circle [1] and I just say "Let's apply the Fourier transform." And I have a few similar rules and gut reactions. You can call it brainstorming, brilliant mathematical intuition, stupid pattern matching or lot of years of experience. I'm not sure and I don't care. Sometimes I fix my idea intermediately because I realize it's wrong. Sometimes some of my coworkers note the error. Sometimes I have to send an email the next day with a retraction. Sometimes the conclusion is wrong but the main idea is correct and I (or someone else) has to fix it [2]. I make a lot of mistakes, but many of my ideas are good enough to get more questions the next week. Is my reasoning better than LLM? I hope so (for now). I sometimes take more time before replying and shut up. It's an important feature. Perhaps LLM can get a feature to write a paragraph in a buffer, something like fake LLM> I have a brilliant idea. Let's try to combine X and Y to solve Z. We first try A and then B and the conclusions is C so ... Wait a minute! Oh! It doesn't work :(. Nevermind. Let's delete this paragraph. And that paragraph is never shown to the user. Is that implemented? Is that a good idea? Perhaps the LLM must tell the idea to another LLM and in both agree show the result to the user. Is there a Stroop Effect in math https://en.wikipedia.org/wiki/Stroop_effect https://en.wikipedia.org/wiki/Stroop_effect ? Perhaps to be as good as a human the LLM must run a few parallel instance and then join the results. --- About hallucinations: I teach Algebra to first year students in the university. We have to explicitly say that with matrices AB is not equal to BA, and in the "hard" course say that det(A+B) is not equal to det(A)+det(B). Are the students hallucinating math properties? (While looking at the draft of some multiple choice midterms, many times I get distracted and use det(2A)=2det(A) :( .) --- [1] It's more complicated. The system must have a circular symmetry, not just items in a circle for a nice drawing. Also sometimes the system has more symmetry and the Fourier Transform is only the easy first step. [2] Sometimes I say "It's obviously true for A, B and C." And when one of my coworkers notice an error in a sign I say. "Then , it's obviously false for A, B and C." Sometimes the A, B and C part is better than my sign calculation.
paulddraper 2y ago
> A human does not do this Have you met humans?
- aprilthird2021 2y ago
  I have never had a human give me some of the answers an LLM has given me, and I've met humans who can't tell basically any country on a map, including the one they live in
dsjoerg 2y ago
> Let me know if you need any further revisions! Fun, you got me :)
burnte 2y ago
> So, to evaluate the intelligence of an LLM based on its first “gut reaction” to a prompt is probably misguided. There's no intelligence to evaluate. They're not intelligent. There's no logic or cogitation in them.
deleted 2y ago
[deleted]
JieJie 2y ago
The US had a president for eight years who was re-elected on his ability to act on his “gut reaction”s. Not saying this is ideal, just that it isn’t the showstopper you present it as. In fact, when people talk about “human values”, it might be worth reflecting on whether this a thing we’re supposed to be protecting or expunging? "I'm not a textbook player, I'm a gut player.” —President George W. Bush. https://www.heraldtribune.com/story/news/2003/01/12/going-to-war-on-a-gut-feeling/28735749007/ https://www.heraldtribune.com/story/news/2003/01/12/going-to...
jrflowers 2y ago
> Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? No. > In fact, if you observe your thinking… There is no reason to believe that LLMs should be compared to human minds other than our bad and irrational tendency towards anthropomorphizing everything. > So, to evaluate the intelligence of an LLM based on its first “gut reaction” to a prompt is probably misguided. LLMs do not have guts and do not experience time. They are not some nervous kid randomly filling in a scantron before the clock runs out. They are the product of software developers abandoning the half-century+ long tradition of making computers output correct answers and chasing vibes instead
- pessimizer 2y ago
  > LLMs do not have guts Just going to ignore the scare quotes then? > do not experience time None of us experience time. Time is a way to describe cause and effect, and change. LLMs have a time when they have been invoked with a prompt, and a time when they have generated output based on that prompt. LLMs don't experience anything, they're computer programs, but we certainly experience LLMs taking time. When we run multiple stages and techniques, each depending on the output of a previous stage, those are time. So when somebody says "gut reaction" they're trying to get you to compare the straight probabilistic generation of text to your instinctive reaction to something. They're asking you to use introspection and ask yourself if you review that first instinctive reaction i.e. have another stage afterwards that relies on the result of the instinctive reaction. If you do, then asking for LLMs to do well in one pass, rather than using the first pass to guide the next passes, is asking for superhuman performance. I feel like this is too obvious to be explaining. Anthropomorphizing things is worth bitching about, but anthropomorphizing human languages and human language output is necessary and not wrong. You don't have to think computer programs have souls to believe that running algorithms over human languages to produce free output that is comprehensible and convincing to humans requires comparisons to humans. Otherwise, you might as well be lossy compressing music without referring to ears, or video without referring to eyes.
  - jrflowers 2y ago
    > Just going to ignore the scare quotes then? Yep. The analogy is bad even with that punctuation. > None of us experience time. That is not true and would only be worthy of discussion if we had agreed that comparing human experience to LLMs predicting tokens was worthwhile (which I emphatically have not done) > You don't have to think computer programs have souls to believe that running algorithms over human languages to produce free output that is comprehensible and convincing to humans requires comparisons to humans. This is true. You also don’t have to think that comparing this software to humans is required. That’s a belief that a person can hold, but holding it strongly does not make it an immutable truth.
NemoNobody 2y ago
Very well said!
AlexandrB 2y ago
> In fact, if you observe your thinking, you might notice that your thought process often takes on different roles and personas. I don't think it's possible to actually observe one's own thinking. A lot of the "eureka" moments one has in the shower, for example, were probably being thought about somewhere in your head but that process is completely hidden from your conscious mind.
- Seb-C 2y ago
  Maybe not observe directly, but analyzing and understanding the thinking process is exactly what psychoanalysis is doing actually. But it takes the help of another person to be able to reach those unconscious thoughts.
ein0p 2y ago
Humans totally do this if their prefrontal cortex shuts down due to fight or flight response. See eg stage fright or giving bullshit answers in leetcode style interviews.
GuB-42 2y ago
No, if I ask a human about something he doesn't know, the first thing he will think about is not a made up answer, it is "I don't know". It actually takes effort to make up a story, and without training we tend to be pretty bad at it. Some people do it naturally, but it is considered a disorder. For LLMs, there is no concept of "not knowing", they will just write something that best matches their training data, and since there is not much "I don't know" in their training data, it is not a natural answer. For example, I asked for a list of bars in a small city the LLM clearly didn't know much about, and gave me a nice list with names, addresses, phone numbers, etc... all hallucinated. Try to ask a normal human to give you a list of bars in a city he doesn't know well enough, and force him to answer something plausible, no "I don't know". Eventually, especially if he knows a lot about bars, you will get an answer, but it absolutely won't be his first thought, he will probably need to think hard about it.
- pessimizer 2y ago
  > No, if I ask a human about something he doesn't know, the first thing he will think about is not a made up answer, it is "I don't know". You've just made this up, through. It's not what happens. How would somebody even know that they didn't know without trying to come up with an answer? But maybe more convincingly, people who have brain injuries that cause them to neglect a side (i.e. not see the left or right side of things) often don't realize (without a lot of convincing) the extent to which this is happening. If you ask them to explain their unexplainable behaviors, they'll spontaneously concoct the most convincing explanation that they can. https://en.wikipedia.org/wiki/Hemispatial_neglect https://en.wikipedia.org/wiki/Hemispatial_neglect https://en.wikipedia.org/wiki/Anosognosia https://en.wikipedia.org/wiki/Anosognosia People try to make things make sense. LLMs try to minimize a loss function.
- IshKebab 2y ago
  Rubbish. Humans absolutely hallucinate things, including lists of bars. Most of them do it infrequently but it absolutely happens. Sometimes they don't even realise it (see all the research about fictitious memories and eye witness accounts). It's definitely a problem that LLMs hallucinate all the time, but let's not pretend humans are perfect and never bullshit. Hell, I've worked with one awful guy who did bullshit just as much as LLMs. Literally never said "I don't know". Always some answer that you were never sure was true or just entirely made up. Annoyingly it worked quite well for him - lots of people couldn't see through it.

by leobg 2y ago

nickpsecurity 2y ago

Our brains also seem to tie our thoughts to observed reality in some way. The parts that do sensing and reasoning interact with the parts that handle memory. Different types of memory exist to handle trade offs. Memory of what makes sense also grows in strength compared to random things we observed. The LLM’s don’t seem to be doing these things. Their design is weaker than the brain on mitigating hallucinations. For brain-inspired research, I’d look at portions of the brain that seem to be abnormal in people with hallucinations. Then, models of how they work. Then, see if we can apply that to LLM’s. My other idea was models of things like the hippocampus applied to NN’s. That’s already being done by a number of researchers, though.

soared 2y ago

I also like comparing a human thought experiment like Einstein (?) would do to forcing an llm to write code to answer a question. Yes you can make a good guess, but making many smaller obvious decisions that lead to an answer is a stronger process.

gus_massa 2y ago

> A human does not do this. You obviously had never asked me anything. (Specialy tech questions while drinking a cup of cofee.) If I had a cent for every wrong answer, I'd be already a millionair.

NemoNobody 2y ago
Why?? To defend AI you used yourself as an example of how we can also be that dumb too. I don't understand. Your example isn't true - what the OP posted is the human condition regarding this particular topic. You, as a human being obviously kno better than to blurt out the first thing that pop into your head - you even have different preset iterations of acceptable things to blurt in certain situations solely to avoid saying the wrong thing like - I'm sorry for your loss. Thoughts and prayers" and stuff like "Yes, Boss" or all the many rules of politeness, all of that is second nature to you, a prevents from blurting shit out. Lastly, how do dumb questions in the mornings with coffee at a tech meeting in any way compare to an AI hallucination?? Did you ever reply with information that you completely made up, has seemingly little to do with the question and doesn't appear to make any logical or reasonable sense as to why that's your answer or how you even got there?? That's clearly not the behavior of an "awake" or sentient thing. That is perhaps the simplest way for normal people to "get it" is by realizing what a hallucination is and that their toddler is likely more capable of comprehending context. You dismissed a plainly stated and correct position, with self depreciating humor - for why?
- gus_massa 2y ago
  Sorry for the long delay... Sometime when I'm working, I see items in a circle [1] and I just say "Let's apply the Fourier transform." And I have a few similar rules and gut reactions. You can call it brainstorming, brilliant mathematical intuition, stupid pattern matching or lot of years of experience. I'm not sure and I don't care. Sometimes I fix my idea intermediately because I realize it's wrong. Sometimes some of my coworkers note the error. Sometimes I have to send an email the next day with a retraction. Sometimes the conclusion is wrong but the main idea is correct and I (or someone else) has to fix it [2]. I make a lot of mistakes, but many of my ideas are good enough to get more questions the next week. Is my reasoning better than LLM? I hope so (for now). I sometimes take more time before replying and shut up. It's an important feature. Perhaps LLM can get a feature to write a paragraph in a buffer, something like fake LLM> I have a brilliant idea. Let's try to combine X and Y to solve Z. We first try A and then B and the conclusions is C so ... Wait a minute! Oh! It doesn't work :(. Nevermind. Let's delete this paragraph. And that paragraph is never shown to the user. Is that implemented? Is that a good idea? Perhaps the LLM must tell the idea to another LLM and in both agree show the result to the user. Is there a Stroop Effect in math https://en.wikipedia.org/wiki/Stroop_effect https://en.wikipedia.org/wiki/Stroop_effect ? Perhaps to be as good as a human the LLM must run a few parallel instance and then join the results. --- About hallucinations: I teach Algebra to first year students in the university. We have to explicitly say that with matrices AB is not equal to BA, and in the "hard" course say that det(A+B) is not equal to det(A)+det(B). Are the students hallucinating math properties? (While looking at the draft of some multiple choice midterms, many times I get distracted and use det(2A)=2det(A) :( .) --- [1] It's more complicated. The system must have a circular symmetry, not just items in a circle for a nice drawing. Also sometimes the system has more symmetry and the Fourier Transform is only the easy first step. [2] Sometimes I say "It's obviously true for A, B and C." And when one of my coworkers notice an error in a sign I say. "Then , it's obviously false for A, B and C." Sometimes the A, B and C part is better than my sign calculation.

paulddraper 2y ago

> A human does not do this Have you met humans?

aprilthird2021 2y ago
I have never had a human give me some of the answers an LLM has given me, and I've met humans who can't tell basically any country on a map, including the one they live in

dsjoerg 2y ago

> Let me know if you need any further revisions! Fun, you got me :)

burnte 2y ago

> So, to evaluate the intelligence of an LLM based on its first “gut reaction” to a prompt is probably misguided. There's no intelligence to evaluate. They're not intelligent. There's no logic or cogitation in them.

deleted 2y ago

[deleted]

JieJie 2y ago

The US had a president for eight years who was re-elected on his ability to act on his “gut reaction”s. Not saying this is ideal, just that it isn’t the showstopper you present it as. In fact, when people talk about “human values”, it might be worth reflecting on whether this a thing we’re supposed to be protecting or expunging? "I'm not a textbook player, I'm a gut player.” —President George W. Bush. https://www.heraldtribune.com/story/news/2003/01/12/going-to-war-on-a-gut-feeling/28735749007/ https://www.heraldtribune.com/story/news/2003/01/12/going-to...

jrflowers 2y ago

> Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? No. > In fact, if you observe your thinking… There is no reason to believe that LLMs should be compared to human minds other than our bad and irrational tendency towards anthropomorphizing everything. > So, to evaluate the intelligence of an LLM based on its first “gut reaction” to a prompt is probably misguided. LLMs do not have guts and do not experience time. They are not some nervous kid randomly filling in a scantron before the clock runs out. They are the product of software developers abandoning the half-century+ long tradition of making computers output correct answers and chasing vibes instead

pessimizer 2y ago
> LLMs do not have guts Just going to ignore the scare quotes then? > do not experience time None of us experience time. Time is a way to describe cause and effect, and change. LLMs have a time when they have been invoked with a prompt, and a time when they have generated output based on that prompt. LLMs don't experience anything, they're computer programs, but we certainly experience LLMs taking time. When we run multiple stages and techniques, each depending on the output of a previous stage, those are time. So when somebody says "gut reaction" they're trying to get you to compare the straight probabilistic generation of text to your instinctive reaction to something. They're asking you to use introspection and ask yourself if you review that first instinctive reaction i.e. have another stage afterwards that relies on the result of the instinctive reaction. If you do, then asking for LLMs to do well in one pass, rather than using the first pass to guide the next passes, is asking for superhuman performance. I feel like this is too obvious to be explaining. Anthropomorphizing things is worth bitching about, but anthropomorphizing human languages and human language output is necessary and not wrong. You don't have to think computer programs have souls to believe that running algorithms over human languages to produce free output that is comprehensible and convincing to humans requires comparisons to humans. Otherwise, you might as well be lossy compressing music without referring to ears, or video without referring to eyes.
- jrflowers 2y ago
  > Just going to ignore the scare quotes then? Yep. The analogy is bad even with that punctuation. > None of us experience time. That is not true and would only be worthy of discussion if we had agreed that comparing human experience to LLMs predicting tokens was worthwhile (which I emphatically have not done) > You don't have to think computer programs have souls to believe that running algorithms over human languages to produce free output that is comprehensible and convincing to humans requires comparisons to humans. This is true. You also don’t have to think that comparing this software to humans is required. That’s a belief that a person can hold, but holding it strongly does not make it an immutable truth.

NemoNobody 2y ago

Very well said!

AlexandrB 2y ago

> In fact, if you observe your thinking, you might notice that your thought process often takes on different roles and personas. I don't think it's possible to actually observe one's own thinking. A lot of the "eureka" moments one has in the shower, for example, were probably being thought about somewhere in your head but that process is completely hidden from your conscious mind.

Seb-C 2y ago
Maybe not observe directly, but analyzing and understanding the thinking process is exactly what psychoanalysis is doing actually. But it takes the help of another person to be able to reach those unconscious thoughts.

ein0p 2y ago

Humans totally do this if their prefrontal cortex shuts down due to fight or flight response. See eg stage fright or giving bullshit answers in leetcode style interviews.

GuB-42 2y ago

No, if I ask a human about something he doesn't know, the first thing he will think about is not a made up answer, it is "I don't know". It actually takes effort to make up a story, and without training we tend to be pretty bad at it. Some people do it naturally, but it is considered a disorder. For LLMs, there is no concept of "not knowing", they will just write something that best matches their training data, and since there is not much "I don't know" in their training data, it is not a natural answer. For example, I asked for a list of bars in a small city the LLM clearly didn't know much about, and gave me a nice list with names, addresses, phone numbers, etc... all hallucinated. Try to ask a normal human to give you a list of bars in a city he doesn't know well enough, and force him to answer something plausible, no "I don't know". Eventually, especially if he knows a lot about bars, you will get an answer, but it absolutely won't be his first thought, he will probably need to think hard about it.

pessimizer 2y ago
> No, if I ask a human about something he doesn't know, the first thing he will think about is not a made up answer, it is "I don't know". You've just made this up, through. It's not what happens. How would somebody even know that they didn't know without trying to come up with an answer? But maybe more convincingly, people who have brain injuries that cause them to neglect a side (i.e. not see the left or right side of things) often don't realize (without a lot of convincing) the extent to which this is happening. If you ask them to explain their unexplainable behaviors, they'll spontaneously concoct the most convincing explanation that they can. https://en.wikipedia.org/wiki/Hemispatial_neglect https://en.wikipedia.org/wiki/Hemispatial_neglect https://en.wikipedia.org/wiki/Anosognosia https://en.wikipedia.org/wiki/Anosognosia People try to make things make sense. LLMs try to minimize a loss function.
IshKebab 2y ago
Rubbish. Humans absolutely hallucinate things, including lists of bars. Most of them do it infrequently but it absolutely happens. Sometimes they don't even realise it (see all the research about fictitious memories and eye witness accounts). It's definitely a problem that LLMs hallucinate all the time, but let's not pretend humans are perfect and never bullshit. Hell, I've worked with one awful guy who did bullshit just as much as LLMs. Literally never said "I don't know". Always some answer that you were never sure was true or just entirely made up. Annoyingly it worked quite well for him - lots of people couldn't see through it.