2 ms·
Hmmm, I might be rounding off wrong? Or reading it wrong? IIUC the data we have: 2K tokens / 12 seconds = 166 tokens/s prefill 120K tokens / (10 minutes == 6
by anoncareer0212 1y ago
Hmmm, I might be rounding off wrong? Or reading it wrong?
IIUC the data we have:
2K tokens / 12 seconds = 166 tokens/s prefill
120K tokens / (10 minutes == 600 seconds) = 200 token/s prefill