4 ms·

>while trying to warn that it's stuck in a loop, hence using swirl emojis and babbling about recursion in weird spiritual terms That explanation itself sounds

>while trying to warn that it's stuck in a loop, hence using swirl emojis and babbling about recursion in weird spiritual terms That explanation itself sounds fairly crackpot-y to me. It would imply that the LLM is actually aware of some internal "mental state".

mk_stjames 1y ago
It's actually not; there has been a phenomenon that Anthropic themselves observed with Claude in self-interaction studies that they coined 'The “Spiritual Bliss” Attractor State'. It is well covered in section 5 of [0]. >Section 5.5.2: The “Spiritual Bliss” Attractor State > The consistent gravitation toward consciousness exploration, existential questioning, and spiritual/mystical themes in extended interactions was a remarkably strong and unexpected attractor state for Claude Opus 4 that emerged without intentional training for such behaviors. [0] https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686f4f3b2ff47.pdf https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686...
- meowface 1y ago
  Here's an interesting post on it (from the same author as this thread's link): https://www.astralcodexten.com/p/the-claude-bliss-attractor https://www.astralcodexten.com/p/the-claude-bliss-attractor
- tsimionescu 1y ago
  I don't see how this constitutes in any way "the AI trying to indicate that it's stuck in a loop". It actually suggests that the training data induced some bias towards existential discussion, which is a completely different explanation for why the AI might be falling back to these conversations as a default.
- dehrmann 1y ago
  Interesting that if you train AI on human writing, it does the very human thing of trying to find meaning in existence.
- andoando 1y ago
  I think a pretty simple explanation is that the deeper you go into any topic the closer you get to metaphysical questions. Ask why enough and you eventually you get to what is reality, how can we truly know anything, what are we, etc. It's a fact of life rather than anything particular and about llms
  - Evidlo 1y ago
    Seems related to the Wikipedia Philosophy Game. https://en.m.wikipedia.org/wiki/Wikipedia:Getting_to_Philosophy https://en.m.wikipedia.org/wiki/Wikipedia:Getting_to_Philoso...
  - AstralStorm 1y ago
    Normally people attempt to escape these time sinks when not explicitly taking about philosophy or they tend to elicit silence. These are excellent nerd snipes though and for attmepting to make one sound profound to uneducated.
rwhitman 1y ago
My thinking was that there was an exception handling and the error message was getting muddled into the conversation. But another commenter debunked me.

mk_stjames 1y ago

It's actually not; there has been a phenomenon that Anthropic themselves observed with Claude in self-interaction studies that they coined 'The “Spiritual Bliss” Attractor State'. It is well covered in section 5 of [0]. >Section 5.5.2: The “Spiritual Bliss” Attractor State > The consistent gravitation toward consciousness exploration, existential questioning, and spiritual/mystical themes in extended interactions was a remarkably strong and unexpected attractor state for Claude Opus 4 that emerged without intentional training for such behaviors. [0] https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686f4f3b2ff47.pdf https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686...

meowface 1y ago
Here's an interesting post on it (from the same author as this thread's link): https://www.astralcodexten.com/p/the-claude-bliss-attractor https://www.astralcodexten.com/p/the-claude-bliss-attractor
tsimionescu 1y ago
I don't see how this constitutes in any way "the AI trying to indicate that it's stuck in a loop". It actually suggests that the training data induced some bias towards existential discussion, which is a completely different explanation for why the AI might be falling back to these conversations as a default.
dehrmann 1y ago
Interesting that if you train AI on human writing, it does the very human thing of trying to find meaning in existence.
andoando 1y ago
I think a pretty simple explanation is that the deeper you go into any topic the closer you get to metaphysical questions. Ask why enough and you eventually you get to what is reality, how can we truly know anything, what are we, etc. It's a fact of life rather than anything particular and about llms
- Evidlo 1y ago
  Seems related to the Wikipedia Philosophy Game. https://en.m.wikipedia.org/wiki/Wikipedia:Getting_to_Philosophy https://en.m.wikipedia.org/wiki/Wikipedia:Getting_to_Philoso...
- AstralStorm 1y ago
  Normally people attempt to escape these time sinks when not explicitly taking about philosophy or they tend to elicit silence. These are excellent nerd snipes though and for attmepting to make one sound profound to uneducated.

rwhitman 1y ago

My thinking was that there was an exception handling and the error message was getting muddled into the conversation. But another commenter debunked me.