3 ms·
Prefill speed and 27B number?
by brcmthrowaway 5mo ago
Prefill speed and 27B number?
- juancn 5mo agoPrefill is around ~600 t/s. I don't remember what the 27B was, I tried a 27B with different quantization at some point for that one, but I settled on the 31B.