4 ms·
no, prompt processing is 512 (look at pp).
by behnamoh 3y ago
no, prompt processing is 512 (look at pp).
- sp332 3y ago512 means the prompt had 512 tokens total. The next column, labelled "t/s", shows tokens per second. The text generation benchmark is labelled "tg 64" because it generates 64 tokens for the benchmark.