4 ms·
Prompt injection is about making the model do something else then specified. Adding words to the text to break the algorithm which does the NLP is more along t
by ffsm8 1y ago
Prompt injection is about making the model do something else then specified.
Adding words to the text to break the algorithm which does the NLP is more along the lines of providing 1 in a boolean field to break the system.
And that's generally something you can mitigate to some degree via heuristics and sanity checking. Doing the same for LLMs is essentially impossible, because it's an effective black box, so you cannot determine the error scenarios and add some mitigations
- IanCal 1y agoIf you don’t think this happens for simpler methods you’ve never deployed them. It’s the exact same problem on a classifier. Have you actually worked with these and are we discussing real world cases?