2 ms·
They're claiming 20+tps inference on a macbook with the unsloth quant.
by regularfry 8mo ago
They're claiming 20+tps inference on a macbook with the unsloth quant.
- embedding-shape 8mo agoYeah, I'm guessing the Mac users still aren't very fond of sharing the time the prefill takes, still. They usually only share the tok/s output, never the input.