4 ms·

What kind of tps are you getting?

• kamranjon 2mo ago
  Generally get 20-25tps - prefill is pretty good around 400-450tps. I have been using compaction at around 100k tokens but mostly just cause it was the default in pi coding agent - might see if i can expand it a bit.
  • Lwerewolf 2mo ago
    I've had it run to ~400k when debugging "obscure" (to it) sequences. Wouldn't recommend more.