> It starts forgetting important parts of the discussion almost straight away, long before the size of the context window becomes a factor. This wasn’t the case

> It starts forgetting important parts of the discussion almost straight away, long before the size of the context window becomes a factor. This wasn’t the case when it was new. Exactly! I remember vividly I was enthralled by GPT-4 because for the first time in my life, I met an entity that was able to take every character of what I wrote into account. It understood the nuances and reason behind each word. It was a breath of fresh air, because even at top companies, written communication is weak as it seems to be interpreted in a lossy manner. It is now almost on the other end of the spectrum where it can't even remember things that I have repeated in effort to help it stick. They had something beautiful, and diluted the shit out of it after they baited you -- which I suppose was the point. I wonder who gets to use the those high quality models now.