In this paper, we showcase the potentially brutal consequences of connecting LLMs to applications (like search). We propose newly enabled attack vectors and tec

In this paper, we showcase the potentially brutal consequences of connecting LLMs to applications (like search). We propose newly enabled attack vectors and techniques and discuss them: - Remote control of chat LLMs - Persistent compromise across sessions - Spread injections to other LLMs - Compromising LLMs with tiny multi-stage payloads - Leaking/exfiltrating user data - Automated Social Engineering - Targeting code completion engines We also provide proof-of-concept demonstrations on GitHub: https://github.com/greshake/lm-safety https://github.com/greshake/lm-safety This paper should be a must-read for anyone that is building a business by integrating LLMs right now. TLDR: Giving an LLM any interface to the outside, like a search capability, has critical security implications. When prompt injection is used and delivered by adversaries instead of the user themself, bad things could happen- and as far as we know, this scenario has not been studied until now.