3 ms·
Seriously nobody thought to use SVD on these weight matrices before?
by atlex2 2y ago
Seriously nobody thought to use SVD on these weight matrices before?
- liuliu 2y agoI did try, but in a wrong way (try to SVD quantization error to recover quality (I.e. SVD(W - Q(W)))). The lightbulb moment in this paper is to do SVD on W and then quantize the remaining.