I think this is the only known benchmark that actually compares different quants: https://oobabooga.github.io/benchmark.html https://oobabooga.github.io/benchma

I think this is the only known benchmark that actually compares different quants: https://oobabooga.github.io/benchmark.html https://oobabooga.github.io/benchmark.html It's not really all that consistent, but larger models can be compressed more without as much loss.