3 ms·
> what about LLMs with seperate instruction and data context? Do such architectures exist? Isn't this separation impossible, for fundamental reasons?
by TeMPOraL 3y ago
> what about LLMs with seperate instruction and data context?
Do such architectures exist? Isn't this separation impossible, for fundamental reasons?
- sebzim4500 3y agoThere's loads of ways you could do this. e.g. cross attention, completely separate tokens for prompt and data, arguably special tokens to surround system prompts (GPT-4 api does this)