3 ms·
The 5.2 tokens per second generation is not that bad, what kills it is the 16.2 prompt processing that makes this too slow to consider even if you have the hard
by throwawayffffas 3mo ago
The 5.2 tokens per second generation is not that bad, what kills it is the 16.2 prompt processing that makes this too slow to consider even if you have the hardware lying around.