3 ms·
I don't understand how this is an "attack"? You can trick a human into copy-pasting something into an LLM and then (somewhat) drive the LLM output? Is the vuln
by ForHackernews 2y ago
I don't understand how this is an "attack"?
You can trick a human into copy-pasting something into an LLM and then (somewhat) drive the LLM output? Is the vuln that humans uncritically believe nonsense chatbots tell them?
- ThrowawayTestr 2y agoThe vuln is that a user can be tricked into exfiltrating data without it being obvious.
- ForHackernews 2y agoBut what "data"? LLMs don't know anything except whatever they were trained on, right?
- probably_wrong 2y agoThe article describes how the content of a document (which in theory should only be sent to OpenAI) can be exfiltrated to an attacker via URL parameters.
- dragonwriter 2y agoLots of LLM applications involve using an LLM to process external data, which makes it part of the prompt. Intuition driven by systems where instruction/code are strictly distinct from data input for processing may be failing you here.
- voiper1 2y agoThe human can "audit" the entire text he's copy-pasting and still miss it. And it provides a way to exfil without it being visible.