4 ms·
The reason I track all those system prompts is due to the way they affect attention when using custom instructions. Having a system message that's already 2,700
by spdustin 3y ago
The reason I track all those system prompts is due to the way they affect attention when using custom instructions. Having a system message that's already 2,700+ tokens before adding custom instructions is a tough hill to climb.