4 ms·
I've switched over to unsloth studio and am hitting those speeds now, thanks for the tip.
by jszymborski 3mo ago
I've switched over to unsloth studio and am hitting those speeds now, thanks for the tip.
- walrus01 3mo agoUnsloth studio is just llama.cpp/llama-server under the hood, so you should see the same performance with the latest daily compiled llama-server (and the right CLI options to load a GGUF file) and any of your own choice of tooling on top of it.