"I can't wait to run this new sota model locally. I'll just use the quantized version that is certain to be better than [other model I'm running]." This is fas

"I can't wait to run this new sota model locally. I'll just use the quantized version that is certain to be better than [other model I'm running]." This is fast becoming one of my top old-man-yells-at-clouds pet peeves. Reported performance metrics are ONLY good for the exact model weights. Quantizing a model, or changing it in any way, requires new evaluation to know how well it performs. Quantizing a good model doesn't mean the quantized version is good.