4 ms·
I set up ollama today and can barely run a 3b parameter model before the lag makes it unbearable. How much is one of these gonna run me?
by tills13 8mo ago
I set up ollama today and can barely run a 3b parameter model before the lag makes it unbearable.
How much is one of these gonna run me?
- deleted 8mo ago[deleted]
- zeta0134 8mo agoI've been pretty happy with my Framework Desktop, though I managed to snag it before RAM prices shot through the roof. Currently, a tricked out model is around $2500. https://frame.work/desktop https://frame.work/desktop Mine sees more use as a Steam machine, but it can run decently large models. Ollama was trivial to get working, and qwen3-coder-next spits out paragraphs of text/code in seconds. I don't really do anything with that, but it's fun to mess around with. (LLMs are still pretty bad at assembly language.)
- jcgrillo 8mo agoYou can buy a 128GB mainboard from framework for $2300, so maybe somewhere a bit over $9k by the time you've got power, storage, cables, racks (they sell those too). I was thinking about getting into one of these Strix Halo setups but decided to go a slightly different route with a lot higher TDP, better throughput, and a bit less VRAM. https://frame.work/products/framework-desktop-mainboard-amd-ryzen-ai-max-300-series?v=FRAFMK0006 https://frame.work/products/framework-desktop-mainboard-amd-...