3 ms·
Exactly. If you've never had an LLM (all models) suddenly start spouting nonsense in a completely different language...you haven't been using LLMs that much. T
by paulddraper 3mo ago
Exactly.
If you've never had an LLM (all models) suddenly start spouting nonsense in a completely different language...you haven't been using LLMs that much. They will go absolutely insane some % of the time.
- andy99 3mo agoWorth looking at https://www.anthropic.com/engineering/a-postmortem-of-three-recent-issues https://www.anthropic.com/engineering/a-postmortem-of-three-... They can “go insane” but it seems often to be infra related as opposed to anything one would consider hallucination. Smaller models will often get stuck repeating a word or phrase forever but that’s a bit different and nobody would call it hallucination.
- ambicapter 3mo agoOne annoying one is we have an LLM-as-a-judge that is supposed to quote parts of a transcript to justify its reasoning, and sometimes it’ll get stuck on something short like “No.” and then just endlessly repeat it: “4. No. 5. No. […] 728. No. […] 1435. No. …”
- unknownfuture 3mo agoI've used LLMs all day five days a week plus my own free time for the last year or so (new job). I've seen plenty of hallucinations and context collapse behaviours. I've never seen that.
- shepherdjerred 3mo agoI've used LLMs quite a lot (Claude, GPT) and have never seen this behavior. You've got something else going on.
- esafak 3mo agoChinese models will do that.