2 ms·
The top local mode in this benchmark is Qwen3.5-9B (Q4_K_M), which is not a big model. 9B = 9 billion parameters. Q4_K_M is the quantization which will come in
by Aurornis 8mo ago
The top local mode in this benchmark is Qwen3.5-9B (Q4_K_M), which is not a big model.
9B = 9 billion parameters. Q4_K_M is the quantization which will come in somewhere around 4.5 bits per weight.
It will run well on a $500 Mac Mini.
- hparadiz 8mo agoI'm actually running it on my AMD 6900 XT right now with 16GB of RAM but looking at my options for upgrading my local model. Can't say I'm a fan of these entry level machines to be honest. I wanna be able to run it with 100k context.