2 ms·
The question is whether this step is actually doing the GPTQ optimized quantization, or simple truncation.
by sp332 3y ago
The question is whether this step is actually doing the GPTQ optimized quantization, or simple truncation.
- sanxiyn 3y agoThis work introduces a new quantization scheme, NF4, for 4-bit NormalFloat, based on previous work on quantile quantization, so it's not a simple truncation, but it's also not a GPTQ-like optimization method. Figure 3 of the paper shows accuracy improvement of NF4 over FP4.
- deleted 3y ago[deleted]