2 ms·
> What are ANY consequential impacts of prompt injection other that the user is able to get information out of the LLM that was put into the LLM? The impact of
by dragonwriter 3y ago
> What are ANY consequential impacts of prompt injection other that the user is able to get information out of the LLM that was put into the LLM?
The impact of prompt injection is provoking arbitrary, unintended behavior from the LLM. If the LLM is a simple chatbot with no tool use beyond retrieving data, that just means “retrieving data different than the LLM operator would have anticipated” (and possibly the user—prompt injection can be done by data retrieved that the user doesn't control, not just the user themselves, because all data processed by the LLM passes through as part of a prompt).
But if the LLM is tied into a framework where it serves as an agent with active tool use, then the blast radius of prompt injection is much bigger.
A lot of the concern about prompt injection isn't about currently popular applications of LLMs, but the applications that have been set out as near term possibilities that are much more powerful.
- simonw 3y agoExactly this. Prompt injection severity varies depending on the application. The biggest risk come from applications that have tool access, but applications that can access private data have a risk too thanks to various data exfiltration tricks.