3 ms·
Ok, but prefill/prompt processing was definitely the weak point before. They were already solid in raw tokens/sec after TTFT
by jtbaker 7mo ago
Ok, but prefill/prompt processing was definitely the weak point before. They were already solid in raw tokens/sec after TTFT