I'd defer to the llama.cpp or exllamav2 repos for quantizing help, I just use their guides myself. For practical use, I'd say searching for the quantized model

I'd defer to the llama.cpp or exllamav2 repos for quantizing help, I just use their guides myself. For practical use, I'd say searching for the quantized model on huggingface.co is the way to go. Chances are high that someone else already did what you're interested in and it's a lot faster to download a pre quantized model. Thebloke, turboderp, lonestriker, bartowski. Between these 4 users on huggingface you can find pretty much every common model already quantized.