4 ms·
Seems this is a common prompt vulnerability pattern: 1. Let Internet content become part of the prompt, and 2. Let the prompt create HTTP requests. With thos
by causal 3y ago
Seems this is a common prompt vulnerability pattern:
1. Let Internet content become part of the prompt, and
2. Let the prompt create HTTP requests.
With those two prerequisites you are essentially inviting the Internet into the chat with you.
- mortallywounded 3y agoYeah-- but it's fun, flirty and exciting in a dangerous way. Kind of like coding in C.
- kfarr 3y agoOr inviting injection attacks by concatenating user data as strings into sql queries in php.
- eichin 3y agoThat's certainly the pattern for the attack, but the vulnerability itself is just "We figured out https://en.wikipedia.org/wiki/In-band_signaling#Telephony https://en.wikipedia.org/wiki/In-band_signaling#Telephony In-band Signalling was a mistake back in the 70s and stopped doing it, chat bots need to catch up"
- causal 3y agoYeah I don't know how you eliminate in-band signalling from an LLM app.
- IshKebab 3y agoI don't think you need to really in this case. Just don't follow links generated by the LLM.
- deleted 3y ago[deleted]
- causal 3y agoThe article demonstrates how the LLM utilized an image to follow the link. Markdown or HTML formatting support is pretty common in chat apps that utilize LLMs.
- IshKebab 3y agoYeah that's what I mean. Downloading an image from a link generated by the LLM is following its link. Just don't do that (unless the same link is present in the source material).
- cronin101 3y agoThe scary part is that > let the prompt create HTTP requests is batteries-included because every language model worth their salt is already able to create markdown and it’s very tempting to utilize this in order to provide layout and break up the wall-of-text output.