4 ms·
TTFT == time to first token. (I would've just said, "the throughput is fantastic, but the latency is about 3 times higher than other offerings".)
by hollerith 1y ago
TTFT == time to first token.
(I would've just said, "the throughput is fantastic, but the latency is about 3 times higher than other offerings".)