2 ms·
Generally get 20-25tps - prefill is pretty good around 400-450tps. I have been using compaction at around 100k tokens but mostly just cause it was the default i
by kamranjon 2mo ago
Generally get 20-25tps - prefill is pretty good around 400-450tps. I have been using compaction at around 100k tokens but mostly just cause it was the default in pi coding agent - might see if i can expand it a bit.
- Lwerewolf 2mo agoI've had it run to ~400k when debugging "obscure" (to it) sequences. Wouldn't recommend more.