4 ms·

Hallucinations are an indication that something is missing, surely? Does chatGPT need a way to verify reality for itself to become truly intelligent ?

superzamp 4y ago
Just the other day, my friend asked me if I knew what this bug she took a picture of was. I replied it looked like a house centipede, and it was indeed one. Two things: 1) My confidence was around 80%, without any sense of absolute certainty. 2) I had no idea why I knew that, and where did I originally learned about what a house centipede looks like.
- ImHereToVote 4y ago
  Yeah but your brain is wet. Try to make a GPU wet, and see how general the problem solving becomes. Check mate.
- TheSpiceIsLife 4y ago
  If you have genuinely no idea why you know that and where you learned it, you should probably seek medical advice concerning maybe early onset dementia or some such. There's a good argument to be made you learned about potentially dangerous insects as a child, from books aimed at children. A reasonable person would make this argument. It's also likely that you retained that information because your would-be peers who aren't around, aren't around because their ascendants inserted their genetic material in to the humanity's descent's to a lesser extent than your extant peers. You might not be able to immediately recall where and when you learned something, and that shouldn't be equated with genuinely having no idea.
- lipoid_ecole 4y ago
  If your friend asked you for proof would you make up an article that doesn't exist and create a link that leads to no where?
ml-anon 4y ago
Calling them "hallucinations" is a shoddy piece of terminology by the community which anthropomorphises them.
- rullelito 4y ago
  We're building human-like intelligence, so this seems quite apt?
  - deleted 4y ago
    [deleted]
  - TheSpiceIsLife 4y ago
    Are we? I'd settle for a poor fascimillie, and argue strongly against human-like. Human-like would be, let's say, a Cylon skin-job. ChatGPT is just a fraking toaster, at best.
- 23B1 4y ago
  What would you recommend calling them? Genuinely curious.
  - pmoriarty 4y ago
    Mistakes.
  - a13o 4y ago
    When humans do it we call them bullshitters. But I agree with GP's eager-anthropomorphization complaint. When algorithms produce verifiably wrong output we call them errors.
  - jebarker 4y ago
    According to Geoff Hinton the correct term from human psychology is "confabulations". Hallucinations are a mistake in perception.
- deleted 4y ago
  [deleted]
vidarh 4y ago
Humans "hallucinate" in the AI sense (it's an awful word that obscures how often we do it) all the time too. You'll find people confidently make claims not supported by data or indeed by any of their own data all the time, where when you poke around in their justifications you'll find they have none. The key difference appears a combination of two factors: We appear to be more likely to have learnt which subjects we're not very knowledgeable about through extensive feedback, be it through school or conversations where we're told we're wrong, and which also would appear to teach us to be more cautious in general. We also appear to have a (somewhat; far from perfect) better ability to separate memory from thoughts about our knowledge. We certainly can go off on wild tangents and make stuff up about any subject, but we get reinforced from very young that there's a time and a place for making stuff up vs. drawing on memory. Both goes back simply to extensive (many years of) reinforcement telling us it has negative effects to make stuff up and/or believe things that aren't true, and yet we still do both of those, just not usually as blatantly as current LLMs without being aware. So I'd expect one missing component is to add a training step that subjects the model to batteries of tests of the limits of their knowledge and incorporating the results of that in the training.
- beanbean01 3y ago
  Yes, but there's a critical difference. When humans hallucinate, it's normally because they misremember something: did you know, for instance, that our brains have to piece every memory together from scratch every time we recall something? It would be a miracle if humans did NOT hallucinate. In other words, the human has a concept of truth or facts and simply had a memory lapse. This thing has no concept of truth at all.
- maister 4y ago
  > Humans "hallucinate" in the AI sense (it's an awful word that obscures how often we do it) all the time too. Agreed. I'd like to add another point to the discussion. It seems to me, as if LLMs are held to a higher standard regarding telling the truth than humans are. In my opinion the reason for this is that computers have been traditionally used to solve deterministic tasks and that people are not used to them making wrong claims.
jebarker 4y ago
Hallucinations are a feature, not a bug. GPT pre-training teaches the model to always produce an answer, even when it has little or not relevant training experience, and on average it does well at that. Part of the point of RLHF in ChatGPT was to teach the model not to hallucinate when it doesn't have good supporting experience encoded in it's weights. This helps but is not perfect. However, it seems like there might be a path to much less hallucinations with more RL training data. As others pointed out, humans hallucinate all the time we just have better training for what level of hallucination is appropriate given supporting evidence and context.

superzamp 4y ago

Just the other day, my friend asked me if I knew what this bug she took a picture of was. I replied it looked like a house centipede, and it was indeed one. Two things: 1) My confidence was around 80%, without any sense of absolute certainty. 2) I had no idea why I knew that, and where did I originally learned about what a house centipede looks like.

ImHereToVote 4y ago
Yeah but your brain is wet. Try to make a GPU wet, and see how general the problem solving becomes. Check mate.
TheSpiceIsLife 4y ago
If you have genuinely no idea why you know that and where you learned it, you should probably seek medical advice concerning maybe early onset dementia or some such. There's a good argument to be made you learned about potentially dangerous insects as a child, from books aimed at children. A reasonable person would make this argument. It's also likely that you retained that information because your would-be peers who aren't around, aren't around because their ascendants inserted their genetic material in to the humanity's descent's to a lesser extent than your extant peers. You might not be able to immediately recall where and when you learned something, and that shouldn't be equated with genuinely having no idea.
lipoid_ecole 4y ago
If your friend asked you for proof would you make up an article that doesn't exist and create a link that leads to no where?

ml-anon 4y ago

Calling them "hallucinations" is a shoddy piece of terminology by the community which anthropomorphises them.

rullelito 4y ago
We're building human-like intelligence, so this seems quite apt?
- deleted 4y ago
  [deleted]
- TheSpiceIsLife 4y ago
  Are we? I'd settle for a poor fascimillie, and argue strongly against human-like. Human-like would be, let's say, a Cylon skin-job. ChatGPT is just a fraking toaster, at best.
23B1 4y ago
What would you recommend calling them? Genuinely curious.
- pmoriarty 4y ago
  Mistakes.
- a13o 4y ago
  When humans do it we call them bullshitters. But I agree with GP's eager-anthropomorphization complaint. When algorithms produce verifiably wrong output we call them errors.
- jebarker 4y ago
  According to Geoff Hinton the correct term from human psychology is "confabulations". Hallucinations are a mistake in perception.
deleted 4y ago
[deleted]

vidarh 4y ago

Humans "hallucinate" in the AI sense (it's an awful word that obscures how often we do it) all the time too. You'll find people confidently make claims not supported by data or indeed by any of their own data all the time, where when you poke around in their justifications you'll find they have none. The key difference appears a combination of two factors: We appear to be more likely to have learnt which subjects we're not very knowledgeable about through extensive feedback, be it through school or conversations where we're told we're wrong, and which also would appear to teach us to be more cautious in general. We also appear to have a (somewhat; far from perfect) better ability to separate memory from thoughts about our knowledge. We certainly can go off on wild tangents and make stuff up about any subject, but we get reinforced from very young that there's a time and a place for making stuff up vs. drawing on memory. Both goes back simply to extensive (many years of) reinforcement telling us it has negative effects to make stuff up and/or believe things that aren't true, and yet we still do both of those, just not usually as blatantly as current LLMs without being aware. So I'd expect one missing component is to add a training step that subjects the model to batteries of tests of the limits of their knowledge and incorporating the results of that in the training.

beanbean01 3y ago
Yes, but there's a critical difference. When humans hallucinate, it's normally because they misremember something: did you know, for instance, that our brains have to piece every memory together from scratch every time we recall something? It would be a miracle if humans did NOT hallucinate. In other words, the human has a concept of truth or facts and simply had a memory lapse. This thing has no concept of truth at all.
maister 4y ago
> Humans "hallucinate" in the AI sense (it's an awful word that obscures how often we do it) all the time too. Agreed. I'd like to add another point to the discussion. It seems to me, as if LLMs are held to a higher standard regarding telling the truth than humans are. In my opinion the reason for this is that computers have been traditionally used to solve deterministic tasks and that people are not used to them making wrong claims.

jebarker 4y ago

Hallucinations are a feature, not a bug. GPT pre-training teaches the model to always produce an answer, even when it has little or not relevant training experience, and on average it does well at that. Part of the point of RLHF in ChatGPT was to teach the model not to hallucinate when it doesn't have good supporting experience encoded in it's weights. This helps but is not perfect. However, it seems like there might be a path to much less hallucinations with more RL training data. As others pointed out, humans hallucinate all the time we just have better training for what level of hallucination is appropriate given supporting evidence and context.