Quantization might need a 'relaxing' or 'normalizing' step before to improve results, it would be an interesting research area Quantize each block or vector as

Quantization might need a 'relaxing' or 'normalizing' step before to improve results, it would be an interesting research area Quantize each block or vector as a unit, bound to some metric instead of simply quantizing the whole thing all at once. It's an interesting topic https://old.reddit.com/r/LocalLLaMA/comments/1ba55rj/overview_of_gguf_quantization_methods/ https://old.reddit.com/r/LocalLLaMA/comments/1ba55rj/overvie...