4 ms·
Before the AI "crisis" it used to take about $3500 to get a prebuilt with a 5090 which can run good enough LLMs. I run reasonable LLMs on just 16GB of VRAM on m
by giancarlostoro 4mo ago
Before the AI "crisis" it used to take about $3500 to get a prebuilt with a 5090 which can run good enough LLMs. I run reasonable LLMs on just 16GB of VRAM on my Mac, and the 5090 has double that.
- dgellow 4mo agoWhat model (and quant) do you run with 16GB? I assume you have a 24GB model, and dedicate 16GB to the model?
- giancarlostoro 4mo agoQuite the opposite, you want something below your max VRAM and you use up to your max VRAM for the context window, so it can run longer. https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-...
- dgellow 4mo agoSorry, I meant a 24GB MacBook Pro model, to run a 16GB LLM :) My use of the term model here is confusing