2 ms·
Are you aware of instruction start and end tags like Mistral has? Do you think that sort of thing has good potential for ignoring instructions outside of those
by luke-stanley 3y ago
Are you aware of instruction start and end tags like Mistral has? Do you think that sort of thing has good potential for ignoring instructions outside of those tags? Small task specific models that aren't instruction following would probably resist most prompt injection types too. Any thoughts on this?
- simonw 3y agoThose are effectively the same thing as system prompts. Sadly they're not a robust solution - models can be trained to place more emphasis on them, but I've never seen a system prompt mechanism like that which can't be broken if the untrusted user input has a long enough length to "trick" the model into doing something else.