3 ms·
Emergent awareness is cool, but it seems Claude Opus has imperfect recall at the 25k token mark, and has lossy recall well before the half-payload mark (100k)?
by ukuina 3y ago
Emergent awareness is cool, but it seems Claude Opus has imperfect recall at the 25k token mark, and has lossy recall well before the half-payload mark (100k)?
For reference, GPT4-Turbo has perfect recall to 64k and drops pretty badly past that until 128k: https://twitter.com/GregKamradt/status/1722386725635580292 https://twitter.com/GregKamradt/status/1722386725635580292