3 ms·
Exactly. Basically when you have a ChatGPT conversation, under the hood this is put at the start of the conversation as "system" instructions to the model. So
by wongarsu 3y ago
Exactly. Basically when you have a ChatGPT conversation, under the hood this is put at the start of the conversation as "system" instructions to the model.
So a typical input might look like
<system> [this prompt]
<user> [what you typed]
<assistant>
And then the LLM is allowed to fill in what the assistant is supposed to say.
- therm0 3y agoSo when the context window rolls over, this system prompt is reinserted?
- immibis 3y agoIt's more likely that it truncates the user prompt.
- wongarsu 3y agoI'd think of it more as keeping state in a separate object, and reassembling the prompt each time as system prompt + the most recent user/assistent conversation pairs + current user prompt. Then you can just shorten the conversation history in the middle to fit your context window. If you are fancy you add a summary of everything you have cut, but I'm not sure if ChatGPT does that.
- ShamelessC 3y agoYes. It's present in every request made.