4 ms·

SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks

54 points by amai 2y ago

handfuloflight 2y ago
Github: https://github.com/arobey1/smooth-llm https://github.com/arobey1/smooth-llm
ipython 2y ago
It concerns me that these defensive techniques themselves often require even more llm inference calls. Just skimmed the GitHub repo for this one and the read me mentions four additional llm inferences for each incoming request - so now we’ve 5x’ed the (already expensive) compute required to answer a query?
padolsey 2y ago
So basically this just adds random characters to input prompts to break jailbreaking attempts? IMHO If you can't make a single-inference solution, you may as well just run a couple of output filters, no? That appeared to have reasonable results, and if you make such filtering more domain-specific, you'll probably make it even better. Intuition says there's no "general solution" to jailbreaking, so maybe it's a lost cause and we need to build up layers of obscurity, of which smooth-llm is just one part.
- ipython 2y ago
  Right. This seems to be the latest in the “throw random stuff at the wall and see what sticks” series of generative ai papers. I don’t know if I’m too stupid to understand or if truly this is just “add random stuff to prompt” dressed up in flowery academic language.
  - pxmpxm 2y ago
    Not surprising - from what I can tell, machine learning has been going down this route for a decade. Anything involving the higher level abstractions (tensor flow / keras /whatever) is full of handwavy stuff about this or that activation function / number of layers / model architecture working the best and doing a trial error with a different component in the above if it doesn't. Closer to kids playing with legos than statistics.
    - malwrar 2y ago
      I’ve actually noticed this in other areas too. Tons of them just swap parts out of existing works, maybe add a novel idea or two, then boom new proposed technique new paper. I remember when I first noticed it after learning to parse the academic nomenclature for a particular subject I was into at the time (SLAM) and feeling ripped off, but hey if you catch up with a subject it’s a good reading shortcut and helps zoom in on new ideas.
mapmeld 2y ago
There are some authors in common with a more recent paper "Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing" https://arxiv.org/abs/2402.16192 https://arxiv.org/abs/2402.16192
freeone3000 2y ago
I find it very interesting that “aligning with human desires” somehow includes prevention of a human trying to bypass the safeguards to generate “objectionable” content (whatever that is). I think the “safeguards” are a bigger problem with aligning with my desires.
- ipython 2y ago
  We’ve seen where that ends up. https://en.m.wikipedia.org/wiki/Tay_(chatbot) https://en.m.wikipedia.org/wiki/Tay_(chatbot)
- wruza 2y ago
  Another question is whether that initial unalignment comes from poor filtering of datasets, or is it emergent from regular, pre-filtered cultured texts. In other words, was an “unaligned” LLM taught bad things from bad people, or does it simply see it naturally and point it out with the purity of a child? The latter would mean something about ourselves. Personally I think that people tend to selectively ignore things too much.
  - GuB-42 2y ago
    We can't avoid teaching bad things to a LLM if we want it to have useful knowledge. For example, you may teach a LLM about nazis, that's expected knowledge. But then, you can prompt a LLM to be a nazi. You can teach it about how to avoid poisoning yourself, but then, you taught it how to poison people. And the smarter the model is, the better it will be at extracting bad things from good things by negation. There are actually training dataset full of bad thing by bad people, the intention is to use them negatively, as to teach the LLM some morality.
    - ujikoluk 2y ago
      Maybe we should just avoid trying to classify things as good or bad.
    - BriggyDwiggs42 2y ago
      But I have no idea why someone might want an LLM to act like a nazi. People read mein kampf in order to study the psychology of a madman and such.
DevotedZest 2y ago
[dead]

54 points by amai 2y ago

Github: https://github.com/arobey1/smooth-llm https://github.com/arobey1/smooth-llm

It concerns me that these defensive techniques themselves often require even more llm inference calls. Just skimmed the GitHub repo for this one and the read me mentions four additional llm inferences for each incoming request - so now we’ve 5x’ed the (already expensive) compute required to answer a query?

padolsey 2y ago

So basically this just adds random characters to input prompts to break jailbreaking attempts? IMHO If you can't make a single-inference solution, you may as well just run a couple of output filters, no? That appeared to have reasonable results, and if you make such filtering more domain-specific, you'll probably make it even better. Intuition says there's no "general solution" to jailbreaking, so maybe it's a lost cause and we need to build up layers of obscurity, of which smooth-llm is just one part.

ipython 2y ago
Right. This seems to be the latest in the “throw random stuff at the wall and see what sticks” series of generative ai papers. I don’t know if I’m too stupid to understand or if truly this is just “add random stuff to prompt” dressed up in flowery academic language.
- pxmpxm 2y ago
  Not surprising - from what I can tell, machine learning has been going down this route for a decade. Anything involving the higher level abstractions (tensor flow / keras /whatever) is full of handwavy stuff about this or that activation function / number of layers / model architecture working the best and doing a trial error with a different component in the above if it doesn't. Closer to kids playing with legos than statistics.
  - malwrar 2y ago
    I’ve actually noticed this in other areas too. Tons of them just swap parts out of existing works, maybe add a novel idea or two, then boom new proposed technique new paper. I remember when I first noticed it after learning to parse the academic nomenclature for a particular subject I was into at the time (SLAM) and feeling ripped off, but hey if you catch up with a subject it’s a good reading shortcut and helps zoom in on new ideas.

mapmeld 2y ago

There are some authors in common with a more recent paper "Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing" https://arxiv.org/abs/2402.16192 https://arxiv.org/abs/2402.16192

freeone3000 2y ago

I find it very interesting that “aligning with human desires” somehow includes prevention of a human trying to bypass the safeguards to generate “objectionable” content (whatever that is). I think the “safeguards” are a bigger problem with aligning with my desires.

ipython 2y ago
We’ve seen where that ends up. https://en.m.wikipedia.org/wiki/Tay_(chatbot) https://en.m.wikipedia.org/wiki/Tay_(chatbot)
wruza 2y ago
Another question is whether that initial unalignment comes from poor filtering of datasets, or is it emergent from regular, pre-filtered cultured texts. In other words, was an “unaligned” LLM taught bad things from bad people, or does it simply see it naturally and point it out with the purity of a child? The latter would mean something about ourselves. Personally I think that people tend to selectively ignore things too much.
- GuB-42 2y ago
  We can't avoid teaching bad things to a LLM if we want it to have useful knowledge. For example, you may teach a LLM about nazis, that's expected knowledge. But then, you can prompt a LLM to be a nazi. You can teach it about how to avoid poisoning yourself, but then, you taught it how to poison people. And the smarter the model is, the better it will be at extracting bad things from good things by negation. There are actually training dataset full of bad thing by bad people, the intention is to use them negatively, as to teach the LLM some morality.
  - ujikoluk 2y ago
    Maybe we should just avoid trying to classify things as good or bad.
  - BriggyDwiggs42 2y ago
    But I have no idea why someone might want an LLM to act like a nazi. People read mein kampf in order to study the psychology of a madman and such.

DevotedZest 2y ago

[dead]