4 ms·
> Is prompt injection even a problem worth worrying about? It depends what API access the AI has. If it's just a chat bot, prompt injection can only reveal fac
by Tv9m 4y ago
> Is prompt injection even a problem worth worrying about?
It depends what API access the AI has. If it's just a chat bot, prompt injection can only reveal facts about its language model. But if the AI has POST access to something, depending on what it is, prompt injection can set off arbitrary human-caused disasters.
- nradov 4y agoThat's not the correct way to do security vulnerability analysis. If an API call can cause a disaster then fix the API. Whether the API consumer is an AI or some other type of system is irrelevant.
- Tv9m 4y ago> If an API call can cause a disaster then fix the API By "API" I'm not referring just to publicly facing REST endpoints. I mean things like shell access for system maintenance, that normally only human professionals like you would be given. In the future it's not clear that humans will be able to dominate that role forever. Hopefully the issues will be recognized while LLM-based agents are still only serving as retrieval systems.