4 ms·

No. I'd say everything the model does is a hallucination. Some writers tend to split ChatGPT oputput into "correct answers" and "Hallucinated answers". If this

No. I'd say everything the model does is a hallucination. Some writers tend to split ChatGPT oputput into "correct answers" and "Hallucinated answers". If this is what you are thinking when saying "Is this part of why ChatGPT hallucinates?", then no. The model never knows if what it is saying is correct. It has a probability about how likely the words are to follow eachother, and that is about it. The temperature changes how these probabilities are calculated, making otherwise less probable words more probable or vice versa. However, the model doesn't choose the most likely word every time. Instead it rolls a dice to see where it wants to go next. If Cheese is 80% and Cookie is 20% it won't necessarily choose Cheese. It rolls a die, and maybe the outcome lands on the less likely Cookie. The roll of the dice happen _after_ temperature has affected the calculation of probabilities. Maybe it is this roll you are thinking of as hallucination?

logicallee 4y ago
You say it never knows if what it is saying is correct, but in my experience it tells you when it does and doesn't know about a concept. For example in one thread I had it give me some random words from linear algebra, then in another thread I combined these into the nonsense phrase "decomposed orthogonal eigenvector division" and asked it if it was familiar with this, and if so to define it. It said it wasn't familiar with it and asked for more information. I'd say that shows it knows what it doesn't know.
detrites 4y ago
While it can be customary to trivialise this, the reality is that no "intelligence" is sure if what it's saying may be correct or constitute any kind of "knowledge". At this level a human can't claim to arbiter fact versus fiction. Fundamentally, no one can be sure what they experience is not some kind of simulation, or a dreamlike state beneath some more real "reality". As such, assigning hierarchy to such a foundation lacking in tangible proof is arbitrary. The point being, unless it's possible to conclusively and infallibly prove you are not hallucinating, you can't lay claim to having knowledge an AI is hallucinating, or not.
- jnwatson 4y ago
  That line of argument swiftly devolves into the case for solipsism. Generally, you may preface any Internet discussion with: first, assume there an independent reality that the author and the readers share. Also causation and the forward arrow of time is true. (It occurs to me that this sounds like prompt engineering.) In other words, you can never conclusively determine you’re not hallucinating (c.f. Hume), but let’s pretend we’re not for the sake of discussion.
  - detrites 4y ago
    Unfortunately, it also swiftly devolves into the issue being discussed... It's not possible to prove whether or not a model can hallucinate or even think, based upon the premise that we have arbitrarily defined what constitutes "thought" (and so also "hallucination"). Can you prove an AI can or can't "think" in a way close enough to us that it could be considered to have basic ("human") rights? What if we built it out of biological material? And would that make any difference? Why? This leads to: how may we objectively prove that the exclusivity of biological "consciousness", as we've defined it, is correct? How may we prove either we or an AI can think, hallucinate, or "be conscious", when we are unable to objectively define the terms in a provable way? I think it may be impossible, and as I said - I don't think it's a trivial concern. We may soon be faced with an entity that demands we respect it, and we need to have worked through as much of that discussion as possible, prior.
    - jnwatson 4y ago
      If you’re a behaviorist, you don’t care if it thinks or the internal machinery at all.
- wodenokoto 4y ago
  > Fundamentally, no one can be sure what they experience is not some kind of simulation, or a dreamlike state beneath some more real "reality". To which I can only reply that I am not sure what I just read ...
- mxkopy 4y ago

logicallee 4y ago

You say it never knows if what it is saying is correct, but in my experience it tells you when it does and doesn't know about a concept. For example in one thread I had it give me some random words from linear algebra, then in another thread I combined these into the nonsense phrase "decomposed orthogonal eigenvector division" and asked it if it was familiar with this, and if so to define it. It said it wasn't familiar with it and asked for more information. I'd say that shows it knows what it doesn't know.

detrites 4y ago

While it can be customary to trivialise this, the reality is that no "intelligence" is sure if what it's saying may be correct or constitute any kind of "knowledge". At this level a human can't claim to arbiter fact versus fiction. Fundamentally, no one can be sure what they experience is not some kind of simulation, or a dreamlike state beneath some more real "reality". As such, assigning hierarchy to such a foundation lacking in tangible proof is arbitrary. The point being, unless it's possible to conclusively and infallibly prove you are not hallucinating, you can't lay claim to having knowledge an AI is hallucinating, or not.

jnwatson 4y ago
That line of argument swiftly devolves into the case for solipsism. Generally, you may preface any Internet discussion with: first, assume there an independent reality that the author and the readers share. Also causation and the forward arrow of time is true. (It occurs to me that this sounds like prompt engineering.) In other words, you can never conclusively determine you’re not hallucinating (c.f. Hume), but let’s pretend we’re not for the sake of discussion.
- detrites 4y ago
  Unfortunately, it also swiftly devolves into the issue being discussed... It's not possible to prove whether or not a model can hallucinate or even think, based upon the premise that we have arbitrarily defined what constitutes "thought" (and so also "hallucination"). Can you prove an AI can or can't "think" in a way close enough to us that it could be considered to have basic ("human") rights? What if we built it out of biological material? And would that make any difference? Why? This leads to: how may we objectively prove that the exclusivity of biological "consciousness", as we've defined it, is correct? How may we prove either we or an AI can think, hallucinate, or "be conscious", when we are unable to objectively define the terms in a provable way? I think it may be impossible, and as I said - I don't think it's a trivial concern. We may soon be faced with an entity that demands we respect it, and we need to have worked through as much of that discussion as possible, prior.
  - jnwatson 4y ago
    If you’re a behaviorist, you don’t care if it thinks or the internal machinery at all.
wodenokoto 4y ago
> Fundamentally, no one can be sure what they experience is not some kind of simulation, or a dreamlike state beneath some more real "reality". To which I can only reply that I am not sure what I just read ...
mxkopy 4y ago