4 ms·
> Gemma 4 31b? Firstly you don't need 64GB for that model. You don't? It for sure doesn't run on my 32 GB M2 MAX.
by vardump 5mo ago
> Gemma 4 31b? Firstly you don't need 64GB for that model.
You don't? It for sure doesn't run on my 32 GB M2 MAX.
- joefourier 5mo agoWhat quant? You should have no problem running it at Q4 with 256K context, Q5 or Q6 even although maybe not at full context. I can run Q4 on a 4090 with just 24GB VRAM.