12 ms·
yes it is 213 tok/s single stream (so per user)
by technoabsurdist 3mo ago
yes it is 213 tok/s single stream (so per user)
- 3836293648 3mo agoSo per subagent*.
- alienbaby 3mo ago*per stream, I guess is more accurate than either?
- unrvl22 3mo agothat 213 wasn't achieved when saturated though. was probably more like 30 tps per stream when doing 2.6k tps.