3 ms·
Does 27b mean it fits one 32GB GPU?
by Muromec 2mo ago
Does 27b mean it fits one 32GB GPU?
- smcleod 2mo agoIt doesn't mean that, but yes it would (with a 5 bit quant).
- kzrdude 2mo ago27B means it has 27 billion parameters, but how that translates to model size depends on the model architecture. For Qwen 3.5 architecture (same as 3.8) the native size is around 55 GB memory. So you need quantization to fit it in 32 GB. Go here: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF https://huggingface.co/unsloth/Qwen3.8-27B-GGUF See the right hand side panel, you see a whole palette of quantizations and their respective sizes. Should give you an idea. Note that these are not the only quantizations available.