4 ms·
I heard that gpt models have about 8k tokens available for context. Does this mean Bing Chat already took few hundres of tokens as the setup prompt?
by est 4y ago
I heard that gpt models have about 8k tokens available for context. Does this mean Bing Chat already took few hundres of tokens as the setup prompt?
- colanderman 4y agoMy first thought too. Chat long enough, Sydney will eventually forget everything about themselves. I wonder if AI chat applications will grow the feature of summarizing their own history prior to running out of context. I was surprised when I learned that GPT models _weren't_ structured to emit a condensed (and opaque) version of the input tokens which could be fed back in as a "context" parameter, e.g. by inserting a "bottleneck" layer between the input tokens and the rest of the network.
- neatze 4y agosurprised, no one has done (seems like I am missing something), it is not complicated to connect multiple chat session in sequence.