2 ms·New SOTA 4-bit weight and KV quantization with optimized GPU performance3 points by Hugsun 2y ago