3 ms·

The 2600 tok/s is an "aggregate", not the actual throughput.

• technoabsurdist 3mo ago
  yes it is 213 tok/s single stream (so per user)
  • 3836293648 3mo ago
    So per subagent*.
    • alienbaby 3mo ago
      *per stream, I guess is more accurate than either?
  • unrvl22 3mo ago
    that 213 wasn't achieved when saturated though. was probably more like 30 tps per stream when doing 2.6k tps.