3 ms·
What model (and quant) do you run with 16GB? I assume you have a 24GB model, and dedicate 16GB to the model?
by dgellow 5mo ago
What model (and quant) do you run with 16GB? I assume you have a 24GB model, and dedicate 16GB to the model?
- giancarlostoro 5mo agoQuite the opposite, you want something below your max VRAM and you use up to your max VRAM for the context window, so it can run longer. https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-...
- dgellow 5mo agoSorry, I meant a 24GB MacBook Pro model, to run a 16GB LLM :) My use of the term model here is confusing