3 ms·

If you feed data to a LLM then there will always be a prompt injection. What you described is limiting the damage that the prompt injection can do, but also its

If you feed data to a LLM then there will always be a prompt injection. What you described is limiting the damage that the prompt injection can do, but also its usefulness.

deleted 3mo ago
[deleted]
arein3 3mo ago
Why is it limiting the usefulness? You have a set of apis that user can access to do something, the llm uses those same apis. How is that limiting usefulness? By not invoking apis user is not allowed to?
- FromTheFirstIn 3mo ago
  The only way to mitigate the damage an LLM can do because of prompt injection is to limit what that LLM can do in the first place. That’s what they mean by limiting its usefulness. If an LLM has access to an api and I want it to abuse that API in some way, I can attack its prompt and eventually get it to use the api the way I want
  - arein3 3mo ago
    All apis have to authorize and authenticate if they do sensitive stuff. Otherwise youre asking for it.
    - NateEag 3mo ago
      Yes, but whatever the LLM has authenticated access to, an attacker can convince it to mess with on their behalf.