4 ms·

yes, was not obvious it's not terabytes per second

• Alifatisk 1y ago
  In the context of LLMs, the unit is token and to measure the output it's tokens per second (T/s)