4 ms·
kordlessagain, thanks for explaining the actual attack vector. I think it's probably some kind of data scrubbing exercise to mitigate the attack. similar to a
by sharemywin 3y ago
kordlessagain, thanks for explaining the actual attack vector.
I think it's probably some kind of data scrubbing exercise to mitigate the attack. similar to a spam filter or something.
- simonw 3y agoBuilding a filter for prompt injection attacks is way harder than you might expect. Nobody has produced a convincing implementation of one of those yet (plenty of people have tried). Read https://llm-attacks.org/ https://llm-attacks.org/ to understand why it's so difficult - that's a paper which generates an unlimited series of weird sequences of tokens which are found to subvert the model.