2 ms·
It has 128k context, but in my experience it's very poor at utilizing it. As in it stops paying attention to large chunks of the context (i.e. doesn't factor th
by 3cats-in-a-coat 3y ago
It has 128k context, but in my experience it's very poor at utilizing it. As in it stops paying attention to large chunks of the context (i.e. doesn't factor them into its answers) unless you specifically point it out.
This affects also GPTs personas. It starts out by following the instructions then gradually reverts to default GPT in the course of the conversation. Then when you ask why's that happening, it's like "oops, sorry, I'm totally <persona> now, see?" And then 5-6 messages later it forgets again.