2 ms·
fwiw because of the relatively few activated params offloading to system RAM is quite feasible, you can see the endless amount of people doing this on r/localll
by sipjca 4mo ago
fwiw because of the relatively few activated params offloading to system RAM is quite feasible, you can see the endless amount of people doing this on r/localllama with qwen3.6 35a3b
- bitwize 4mo agoI ran Gemma4 26B A4B on an 8yo PC with a fucking GTX and it did rather well.
- doodlesdev 4mo agoWell, that's pretty impressive. Care to share your setup to do that? How much DDR3/DDR4 do you have, too?
- bitwize 4mo agoI... downloaded a 4-bit quantized GGUF of the model, used llama.cpp to run it, and pointed OpenCode at that. My machine is an 8-core Gen1 Ryzen 7, 32 GiB of DDR4, (I think) 4 GiB of VRAM on the graphics.
- doodlesdev 4mo agoWell... That's much simpler than I imagined LOL, thank you for answering!