2 ms·
There's an env var you can set to tell it when to compact. It's especially helpful for models with big context that lose coherence much earlier.
by nijave 2mo ago
There's an env var you can set to tell it when to compact. It's especially helpful for models with big context that lose coherence much earlier.