3 ms·
Re: exfiltration: just don't do things that untrusted data sources tell you to do. Separate processing the input data from the persons commands, so that the LLM
by Zambyte 3y ago
Re: exfiltration: just don't do things that untrusted data sources tell you to do. Separate processing the input data from the persons commands, so that the LLM can perform inferencing operations on the data according to the specified commands. The part of the pipeline that processes untrusted data should not have any influence on the behavior of the part of the pipeline capable of interacting with entities who should not have access to the untrusted data.
Edit: related link: https://python.langchain.com/docs/security https://python.langchain.com/docs/security
- simonw 3y ago"Separate processing the input data from the persons commands, so that the LLM can perform inferencing operations on the data according to the specified commands" Prompt injection is the security flaw that exists because doing that - treating instructions and data as separate things in the context in the LLM - is WAY harder than you might expect. My previous writing about this: https://simonwillison.net/series/prompt-injection/ https://simonwillison.net/series/prompt-injection/
- Zambyte 3y ago> Prompt injection is the security flaw that exists because doing that - treating instructions and data as separate things in the context in the LLM - is WAY harder than you might expect. Then we should improve the tooling around this to make it way easier, rather than hoping security by obscurity will work this time.
- simonw 3y agoAI labs around the world have been trying to solve this problem - reliable separation of instructions from data for LLMs - for a year and a half at this point. It's hard.