5 ms·

I just want the 100k context window Anthropic gave Claude, but for GPT. Claude likes to hallucinate when I ask it to build chapter notes for a class I’m taking

• cjonas 3y ago
  The large context is the reason its hallucinating and gpt would likely have the same problem. You can see this effect even on gpt-3.5-turbo-16k. Performance and accuracy are directly correlated with context size
  • danielbln 3y ago
    That's just GPT3.5 having a pretty spotty attention. This becomes much less of a problem if you use gpt-4-32k.
• Obscurity4340 3y ago
  *confabulate. Hallucination is StableDiffusion
• behnamoh 3y ago
  The lost-in-the-middle phenomenon makes me _not_ want larger context windows because the model cannot effectively use it. It seems like we need another arch to process large ctx size.
• msp26 3y ago
  Performance on long contexts just isn't very good right now. Relevant paper: Lost in the Middle: How Language Models Use Long Contexts https://arxiv.org/abs/2307.03172 https://arxiv.org/abs/2307.03172