8 ms·
The vulnerability is not limited to Custom GPTs, that was just the latest example of an exploit vector and demo. Anytime untrusted data is in the chat context
by wunderwuzzi23 3y ago
The vulnerability is not limited to Custom GPTs, that was just the latest example of an exploit vector and demo.
Anytime untrusted data is in the chat context (e.g. reading something from a website, processing an email via a plugin, analyzing a PDF, uploading an image,..) instructions in the data can perform this attack.
It's a data exfil technique that many LLM applications suffer from.
Other vendors fixed it a while ago after responsible disclosure (e.g Bard, Bing Chat, Claude, GCP, Azure,...), but OpenAI hadn't yet taken action since being informed about it in April 2023.
For example, here are the details on the Bing Chat exploit and how Microsoft fixed it: https://embracethered.com/blog/posts/2023/bing-chat-data-exfiltration-poc-and-fix/ https://embracethered.com/blog/posts/2023/bing-chat-data-exf... - and many other examples on https://embracethered.com https://embracethered.com