2 ms·
Do you pay for the model? Or run it locally? If so, which hardware do you use? Is it fast?
by joelthelion 4mo ago
Do you pay for the model? Or run it locally? If so, which hardware do you use? Is it fast?
- kstenerud 4mo agoI use an RTX 3090 with 24gb on it, which can comfortably fit https://huggingface.co/Qwen/Qwen3.6-27B https://huggingface.co/Qwen/Qwen3.6-27B. It's freely downloadable, so you only pay for the gfx card and the electricity. There are also plenty of models out there that work well on Apple unified memory. But mostly I just use Claude now since I already have a max subscription. I've done hybrid setups in the past where I had Claude do the hard stuff and a local LLM do the code monkey stuff, but with the right optimizations I'm no longer hitting the Claude token ceiling.
- joelthelion 4mo agoThank you !