3 ms·
So an initial LLM takes trusted input and a list of tools, and puts together an executable Python script using those tools. Some of those tools use LLMs for ext
by petesergeant 2y ago
So an initial LLM takes trusted input and a list of tools, and puts together an executable Python script using those tools. Some of those tools use LLMs for extraction purposes from downstream data, but the downstream LLMs don’t have access to tool usage, so even if the data to evaluate has malicious data, the worst thing they can return is a misleading string that’s not re-evaluated by an LLM, it’s simply set in a Python variable.
This feels like a lot of engineering for quite a narrow mitigation, and I guess I’m a little surprised to see a paper on it. Perhaps I need to start writing up some of my own techniques!
- mentalgear 2y agoDefinitely, I'd be interested even if you could just outline them!
- petesergeant 2y agoHere is one I wrote today on LLMs that can handle chat input like humans write: multiple disjointed messages arriving asynchronously that need to be treated as one: https://sgnt.ai/p/interruptible-llm-responses/ https://sgnt.ai/p/interruptible-llm-responses/ I use a similar technique to the article for trying to avoid jailbreaks by putting untrusted input through zod to check I got back a JSON structure of the right shape, which has been very effective. I’ve been sprinkling lexical in-memory search throughout prompts to save inference calls, which has been very effective