2 ms·
Interesting experiment! However, I think it lacks the most interesting information - which is a latency in tokens/sec and how it decays with increasing of para
by ekropotin 1y ago
Interesting experiment!
However, I think it lacks the most interesting information - which is a latency in tokens/sec and how it decays with increasing of parallelism.
I also not exactly got what it has to do with AI Agents in particular.