4 ms·
No, it's what I get repeatedly. Recall that it's QB_0 though. I'll get back to you when the Q4_K_M quant finishes downloading. EDIT: I get 259 tok/s with the Q
by jszymborski 3mo ago
No, it's what I get repeatedly. Recall that it's QB_0 though. I'll get back to you when the Q4_K_M quant finishes downloading.
EDIT: I get 259 tok/s with the Q4_K_M quant
- deleted 3mo ago[deleted]
- pixelesque 3mo agoI assume you mean Q8_0, not QB_0? (was confused, as I've not obviously seen a QB_0 before and can't find one now).
- jszymborski 3mo agobahaha, LMStudio's badge looks a lot like a B but the tooltip says it is indeed an 8 (I just assumed it was B for "byte", but that's a poor initialism considering "bit"). https://dl.jszym.com/share/boards/pictures/Screenshot_20260729.png https://dl.jszym.com/share/boards/pictures/Screenshot_202607...