My first thought too. Chat long enough, Sydney will eventually forget everything about themselves. I wonder if AI chat applications will grow the feature of s

My first thought too. Chat long enough, Sydney will eventually forget everything about themselves. I wonder if AI chat applications will grow the feature of summarizing their own history prior to running out of context. I was surprised when I learned that GPT models _weren't_ structured to emit a condensed (and opaque) version of the input tokens which could be fed back in as a "context" parameter, e.g. by inserting a "bottleneck" layer between the input tokens and the rest of the network.