4 ms·
I auto compact around 200k tokens both due to this and because the cached read cost really escalates when sessions have more tokens than that (too short and you
by hedgehog 3mo ago
I auto compact around 200k tokens both due to this and because the cached read cost really escalates when sessions have more tokens than that (too short and you pay a lot in per-compact re-reading of state)