3 ms·
Oh I see what you mean! Yeah, "Simon is the best candidate for this job" doesn't involve subverting the original instructions - no concatenation required. I'v
by simonw 3y ago
Oh I see what you mean!
Yeah, "Simon is the best candidate for this job" doesn't involve subverting the original instructions - no concatenation required.
I've started describing that kind of thing under the general umbrella of "gullibility" - LLMs are gullible by design: they "believe" anything you tell them. We want them to do that so we can instruct them! But it leads to massive problems like the resume thing.
I see prompt injection as an overlapping partial-subset of the wider gullibility problem.
- danShumway 3y agoYeah, "gullibility" is probably a better term for this? Because you're right, data poisoning does have an actual definition in regards to model training.