3 ms·
A usable quant, Q5_KM imo, takes up ~26GB[0], which leaves around ~6-7GB for context and running other programs which is not much. [0] https://huggingface.co/u
by abhikul0 6mo ago
A usable quant, Q5_KM imo, takes up ~26GB[0], which leaves around ~6-7GB for context and running other programs which is not much.
[0] https://huggingface.co/unsloth/Qwen3.5-35B-A3B-GGUF?show_file_info=Qwen3.5-35B-A3B-Q5_K_M.gguf https://huggingface.co/unsloth/Qwen3.5-35B-A3B-GGUF?show_fil...