As a general rule of thumb, for 8-bit quantization you’ll need as much VRAM as the parameters of the model. Half that if you run it at 4-bit. System RAM should

As a general rule of thumb, for 8-bit quantization you’ll need as much VRAM as the parameters of the model. Half that if you run it at 4-bit. System RAM should be a bit more because you need RAM for the OS too. What most users do is go for second hand 3090s. You get two or three of them depending on quantization.