3 ms·
Well, the weights are accumulated in full precision and are multiplied by a full-precision scale factor after quantization, and the activations and backward pas
by yorwba 8mo ago
Well, the weights are accumulated in full precision and are multiplied by a full-precision scale factor after quantization, and the activations and backward pass are computed in full precision as well, so it's not quite true 4-bit precision training. The resulting model can be stored with just slightly more than 4 bits per parameter, though.
- jcalvinowens 8mo agoI really just don't understand how the quantization error doesn't ruin the results. Is there some reading you'd recommend? I can easily understand how the block formats win.
- shivnathtathe 8mo ago[dead]