11 ms·
We’re supposed to be fixing LLM security by adding a non-LLM layer to it, not adding LLM layers to stuff to make them inherently less secure. This will be a n
by DANmode 6mo ago
We’re supposed to be fixing LLM security by adding a non-LLM layer to it,
not adding LLM layers to stuff to make them inherently less secure.
This will be a neat concept for the types of tools that come after the present iteration of LLMs.
Unless I’m sorely mistaken.
- SkyPuncher 6mo agoDefense in depth. Layers don't inherently make something less secure. Often, they make it more secure.
- yakkomajuri 6mo agoI do think this is likely to make things more secure but it's also dangerous by potentially giving users a false sense of complete security when the security layer is probabilistic rather than deterministic. EDIT: it does seem to have a deterministic layer too and I think that's great
- reassess_blind 6mo agoIt looks as if this tool has traditional static rules to allow/deny requests, as well as a secondary LLM-as-a-judge layer for, I imagine, the kinds of rules that would be messy or too convoluted to implement using standard rules.
- stingraycharles 6mo agoI think the parent’s point is that this should be implemented using e.g. Bayesian statistics rather than an LLM, as the judge LLM is vulnerable to the exact same types of attacks that it’s trying to protect against. Most proper LLM guardrails products use both.
- snug 6mo agoI think this can be great as additional layer of security. Where you can have a non llm layer do some analysis with some static rules and then if something might seem phishy run it through the llm judge so that you don’t have to run every request through it, which would be very expensive. Edit: actually looks like it has two policy engines embedded
- windexh8er 6mo agoAnd we don't think the judge can/will be gamed? Also... It's an LLM, it's going to add delay and additional token burn. One subjective black box protecting another subjective black box. I mean, what couldn't go wrong?
- lukewarm707 6mo agoyou can use a safety model trained on prompt injections with developer message priority. user message becomes close to untrusted compared to dev prompt. also post train it only outputs things like safe/unsafe so you are relatively deterministic on injection or no injection. ie llama prompt guard, oss 120 safeguard.
- windexh8er 6mo agoUnfortunately it's not that simple. Self-policing AI systems will always be gamed. Just one [0] example of this among many. [0] https://www.hiddenlayer.com/research/same-model-different-hat https://www.hiddenlayer.com/research/same-model-different-ha...
- ImPostingOnHN 6mo agoWhat happens when a prompt injection attack exploits the judge LLM and results in a higher level of attacker control than if it never existed?
- vova_hn2 6mo agoHow can it result in a higher level of control? I don't see why the "judge" should have access to anything except one tool that allows it to send an "accept" or "deny" command.
- nl 6mo ago> We’re supposed to be fixing LLM security by adding a non-LLM layer to it, If people said "we build a ML-based classifier into our proxy to block dangerous requests" would it be better? Why does the fact the classifier is a LLM make it somehow worse?
- Retr0id 6mo agoThe fact that LLMs are "smarter" is also their weakness. An oldschool classifier is far from foolproof, but you won't get past it by telling it about your grandma's bedtime story routine.
- reassess_blind 6mo agoFairly hard to bypass the latest LLMs with grandma's bedtime story these days, to be fair.
- Retr0id 6mo agoThat specific trick yes, but the general concept still applies.
- reassess_blind 6mo agoIt does, but it's certainly not trivial. In fact there's an unclaimed $1000 bounty on prompt injecting OpenClaw: https://hackmyclaw.com/ https://hackmyclaw.com/
- DANmode 6mo agoIs that enough?
- reassess_blind 6mo agoEnough for what?
- 6mo ago