3 ms·
> a "low end" desktop card like the 3070 which has only 8GB VRAM I'm finding that my 12GB 3060 is not enough to run even LLaMA-7B without CPU offload, thank Ar
by LoganDark 4y ago
> a "low end" desktop card like the 3070 which has only 8GB VRAM
I'm finding that my 12GB 3060 is not enough to run even LLaMA-7B without CPU offload, thank Arceus for quantization...
- kkielhofner 4y agoGive it time - there's a lot of incredible work being done here and we'll likely eventually see some kind of black magic to make it possible.