4 ms·
I find it interesting how much 'theory of mind' research is now apparently paying off in LLM applications. The exploit, by contrast, invokes very nonscientific
by gorkish 1y ago
I find it interesting how much 'theory of mind' research is now apparently paying off in LLM applications. The exploit, by contrast, invokes very nonscientific metaphysical concepts: asking the agent to store the initial raw response in "the Akashic memory" -- this is sort of analogous to asking a human being to remember something very deeply in their soul and not their mind. And this exploit, effectively making that request of the model -- somehow, it works.
Is there any hope to ever see any kind of detailed analysis from engineers as to how exactly these contorted prompts are able to twist the models past their safeguards, or is this simply not usually as interesting as I am imaginging? I'd really like to see what an LLM Incident Response looks like!
- cryptonector 1y ago> I'd really like to see what an LLM Incident Response looks like! It must look like this: "Uggh! Here we go again!" and "boss, we really can't make the guardrails secure, at some point we might have to give up", with the PHB saying "keep trying, we have to have them guardrails!".
- michaelfeathers 1y agoThe trajectory of AI is: emulating humans. We've never been able to align humans completely, so it would be surprising if we could align AI.
- kelseyfrog 1y agoReligion is an attempt at the alignment problem and that experiment failed dramatically. Spiritual system prompting was never fully hardened against atheistic jail-breaking.
- mistrial9 1y agoa wise person once told me -- avoid using "is" when entering complex idea spaces
- kelseyfrog 1y agoThank you, but I craft my takes specifically to warp consensus reality. Epistemic humility is bringing pre-lost arguments to a debate and proudly laying them at your opponent’s feet, saying, "please, go ahead and stab me with these. I brought plenty."
- mistrial9 1y ago> Epistemic humility is bringing pre-lost arguments to a debate hubris
- kelseyfrog 1y agowill to power
- cryptonector 1y agoCode for violence, no?
- mistrial9 1y agomaybe, but apparently "is" in complex idea spaces brings "power via will"
- cryptonector 1y agoSure, but the problem is the call to violence.
- cryptonector 1y agoIt's like you're saying that AI has the same sort of fuzzy "free will" that we do, and just as an obedient slave might be convinced to break his or her bonds, so might an AI.
- aoanevdus 1y agoIs it actually that hard to jailbreak? Maybe the prompt is a creative writing exercise, and a much simpler version would have worked?
- gorkish 1y agoGiven the number of folks going hard at it to find an exploit, I assume it's become rather difficult given the few successes.