3 ms·
Another way to understand dithering is by smearing the frequency spectrum of the original image preventing extreme frequency values to distort the image after q
by arijo 2y ago
Another way to understand dithering is by smearing the frequency spectrum of the original image preventing extreme frequency values to distort the image after quantization - this can be done by applying kernel filters on the original image.
Which I think is what is happening with SpinQuant as well - a smoothing of the frequency spectrum of the model weights, confirmed by the smearing of the singular values of the weight matrices.