4 ms·

So is this how hallucinations form ? It proposes something half-baked, and then later refers back to it as fact ?

• abraxas 4y ago
  Sounds like we already have full blown cyber replicas of politicians then.
  • mythrwy 4y ago
    Or some bosses I've had. I actually can't wait to start reporting to GPT, how much crazier can it be?
• zerocrates 4y ago
  Some of this is probably just an artifact of how ChatGPT specifically works: I believe I have it correct that it basically feeds the transcript of the conversation, to the extent possible, back to itself as part of the prompt going forward. So its prior responses in the session are part of the text it's generating from.
• HarHarVeryFunny 4y ago
  No, it's generating output word by word, not planning very far ahead (it can't since it doesn't even know what words it's really going to generate, since they are randomly sampled), and essentially backs itself into a corner where completing the train of thought requires a fact that it doesn't actually have. Just as a made up example, say you asked "what is the capital city of england", and the model had seen similar questions in it's training data that were answered with "the capital city of X is Y", so it starts word-by-word generating this type of response "the capital city of england is", but it then turns out the model doesn't actually know the answer (i.e. this partial response context doesn't cause it to predict the correct answer), so it blithley predicts the next word as whatever it's inner machinations come up with, maybe "buckingham palace" or "flavor town". "Hallucination" seems a poor way to describe it, nor is it lying since there's no bad intent ... it's basically "starting to speak before brain engaged", a bit like a game show contestant being a bit too enthusiastic and hitting the "answer" button without actually having a fully thought out answer in mind.