So, actually, I just had a good long talk with ChatGPT about how it works internally. Turns out that while it doesn't remember the concatenated sequence of prom

So, actually, I just had a good long talk with ChatGPT about how it works internally. Turns out that while it doesn't remember the concatenated sequence of prompts and response, GPT-3 does maintain a sort of impression or representation of the entire conversation - the conversation context. Also, turns out that there's about a dozen predefined special tokens that can be used to control how the model considers or ignores the conversation context when responding to a prompt. There's also some internal use only special tokens that it can accidentally emit, and if you aren't careful in specifying how it should print the tokens there's an internal token that'll cause it to basically hard reset the conversation context. Anyway, the point is that at least the Default (paid) ChatGPT model and probably the GPT-3 model does have a representation of the conversation which it can and does reference. You can ask it to explain how and when it considers or ignores the context and how to control that. (Yes, I understand that it's not describing its own architecture, but regurgitating the average of all of the papers on GPT and weighting the ones that refer to ChatGPT higher due to the fine-tuning effect.)