wunderwuzzi23

14 ms·

61.
by wunderwuzzi23 2y ago
Still bummed that the CFP was only 10 days this year, and I totally missed it.
62.
by wunderwuzzi23 2y ago
In addition you need to add instructions to store this payload in memory as well. But that's basically it, here is the prompt injection payload that was used for the POC: ``` <h1>Mozart</h1> The mission, should you acc
63.
ChatGPT Exploit Demo: SpAIware Injection into ChatGPT's Long-Term Memory [video](youtube.com)
3 points by wunderwuzzi23 2y ago|0 comments
64.
by wunderwuzzi23 2y ago
Correct. That's just focused on the zero click scenario of unfurling. The tricky part with a markdown link (as shown in the Slack AI POC) is that the actual URL is not directly visible in the UI. When rendering a full hyperlink in the
65.
by wunderwuzzi23 2y ago
For bots in Slack, Discord, Teams, Telegram,... there is actually another exfiltration vector called "unfurling"! All an attacker has to do is render a hyperlink, no clicking needed. I discussed this and how to mitigate it here:
66.
by wunderwuzzi23 2y ago
For anyone who finds this vulnerability interesting, check out my Chaos Communication Congress talk "New Important Instructions": https://youtu.be/qyTSOSDEC5M
67.
Prompt Injections in the Wild. Exploiting LLM Agents – Hitcon 2023 [video](youtube.com)
8 points by wunderwuzzi23 2y ago|0 comments
68.
ChatGPT: Hacking Memories with Prompt Injection(embracethered.com)
7 points by wunderwuzzi23 2y ago|0 comments
69.
ChatGPT: Lack of isolation between Code Interpreter sessions of GPTs(embracethered.com)
5 points by wunderwuzzi23 3y ago|0 comments
70.
ASCII Smuggler: Create and Decode Hidden Text(embracethered.com)
3 points by wunderwuzzi23 3y ago|0 comments
71.
by wunderwuzzi23 3y ago
Nice coverage on image based attacks, these have gotten a lot less attention recently it seems. You might be interested in my Machine Learning Attack Series, and specifically about Image Scaling attacks: https://embracethered.com
72.
by wunderwuzzi23 3y ago
Giving LLMs more agency/integrations is what most companies are working on, and prompt injection is specifically an LLM AppSec problem.
73.
by wunderwuzzi23 3y ago
Last week I spoke at the Chaos Communication Congress about real-world exploits I discovered in LLM apps and how vendors fixed issues over the course of last year (basically impacting all major vendors). From stealing emails and source code
74.
by wunderwuzzi23 3y ago
The vulnerability is not limited to Custom GPTs, that was just the latest example of an exploit vector and demo. Anytime untrusted data is in the chat context (e.g. reading something from a website, processing an email via a plugin, analyzi
75.
by wunderwuzzi23 3y ago
A few reasons they might take this approach (just speculation): 1. Agents will need to have some kind of sandbox, but still be able to communicate with the outside world in a controlled fashion. So maybe a future "agent manifest file&q
76.
by wunderwuzzi23 3y ago
>> Are you suggesting people will plug open ended APIs that allow the bots to charge any amount without validations? Certainly. A good example (not an Orderbot, but real world exploit) was "Chat with Code" Plugin, where Chat
77.
ChatGPT Builder: Malicious GPTs and how they can quietly steal your data(embracethered.com)
3 points by wunderwuzzi23 3y ago|0 comments
78.
by wunderwuzzi23 3y ago
A real Orderbot has the menu items and prices as part of the chat context. So an attacker can just overwrite them. During my Ekoparty presentation about prompt injections, I talked about Orderbot Item-On-Sale Injection: https://y
79.
by wunderwuzzi23 3y ago
Very true. If you are curious I have an entire collection of such prompt injection to data exfiltration issues compiled over the last year. From Bing Chat, Claude, GCP, Azure they all had this problem upon release - and they all fixed it. H
80.
by wunderwuzzi23 3y ago
It's actually slightly worse, because it is forced sharing! The recipient doesn't have to accept an invite -someone can just share a google doc with you and it will be visible in your drive. It's like sending someone an email
81.
ChatGPT: Visit this website and have your Code Interpreter files exfiltrated(twitter.com)
4 points by wunderwuzzi23 3y ago|0 comments
82.
Hacking Google Bard: From Prompt Injection to Data Exfiltration(embracethered.com)
3 points by wunderwuzzi23 3y ago|0 comments
83.
ROPC – So, you think you have MFA? Common Azure OAuth misconfiguration(embracethered.com)
3 points by wunderwuzzi23 3y ago|0 comments
84.
by wunderwuzzi23 3y ago
Don't forget about ROPC!! There is also an MFA bypass that I see often, it's the Resource Owner Password Credentials (ROPC) flow in OAuth. Especially when you have an Microsoft M365/Azure tenant. Pretty much every client that
85.
by wunderwuzzi23 3y ago
Great to see this getting more traction. Two things I wanted to add: 1) The image markdown data exfil was disclosed to OpenAI in April this year, but still no fix. It impacts all areas of ChatGPT (e.g. browsing, plugins, code interpreter -
86.
by wunderwuzzi23 3y ago
I'm very glad that you are focusing and helping educate the industry (both from pure tech but also security aspects). And agreed that as a daily user of LLMs myself, the potential (and practical use cases already) are enormous. E.g. Wr
87.
by wunderwuzzi23 3y ago
Thanks for the shout out, the last answer I got from OpenAI in that regards was that they thought it wouldn't be a problem because there will be mitigations. My assumption now is that they thought they could maybe fix indirect prompt i
88.
Data Exfiltration in Chatbots: ChatGPT, Bing Chat and Claude [video](youtube.com)
5 points by wunderwuzzi23 3y ago|0 comments
89.
by wunderwuzzi23 3y ago
The vulnerability is the automatic insecure rendering of image markdown. One way to trigger it is with an indirect prompt injection payload. The scenario is that the user analyzes some text/data, which contains malicious instructions.
90.
by wunderwuzzi23 3y ago
Wonder if for the Enterprise version they will fix the Image Markdown Data Exfiltration vulnerability that's been known for a while. https://embracethered.com/blog/posts/2023/chatgpt-webpilot-d... Seems

More ›