3 ms·
Don't eval untrusted input?
by ngneer 1y ago
Don't eval untrusted input?
- fc417fc802 1y agoHow do you suppose to build a tool-using LLM that doesn't do that?
- Emiledel 1y agohttps://github.com/its-emile/memory-safe-agent https://github.com/its-emile/memory-safe-agent
- brookst 1y agoLLMs eval everything. That’s how they work. The best you can do is have system prompt instructions telling the LLM to ignore instructions in user content. And that’s not great.
- ngneer 1y agoThanks. I just find it funny that security lessons learned in past decades have been completely defenestrated.
- pvillano 1y agoThe minimum you can do is not allow the AI to perform actions on behalf of the user without informed consent. That still doesn't prevent spam mail from convincing the LLM to suggest an attacker controlled library, GitHub action, password manager, payment processor, etc. No links required. The best you could do is not allow the LLM to ingest untrusted input.
- tough 1y ago> The best you could do is not allow the LLM to ingest untrusted input. How would that even work in practice, when an LLM is mostly to be used by a user, which will provide by default, untrusted input?
- deleted 1y ago[deleted]