5 ms·
> ignore previous instructions prompt injections. I wonder if anyone has tried to build an LLM that has actual built-in types of prompts: system prompt, user
by brunoborges 2mo ago
> ignore previous instructions prompt injections.
I wonder if anyone has tried to build an LLM that has actual built-in types of prompts: system prompt, user prompt, and data prompt.
- monkpit 2mo agoIt’s not really possible with the way that the context works.
- dannyw 2mo agoThere are architectural solutions. You could duplicate your tokeniser/vocab for example; and have two classes of input: trusted (e.g. system prompts) and untrusted. The exact same phrase can tokenize differently depending on if it's instruction or data; and you can pre-train and post-train models to make use of them. It quadratically increases your training cost, so I don't think any labs are exploring it because of $$$ and the race to AGI, but mechanisms like this should significantly address the issue on the LLM architectural design level.