4 ms·
The negative quality impact of quantization is more pronounced for smaller models [0], so I'm surprised this tiny quant works at all. [0] or rather models clos
by Palmik 2y ago
The negative quality impact of quantization is more pronounced for smaller models [0], so I'm surprised this tiny quant works at all.
[0] or rather models closer to saturation, which is a function of model params and amount of training
- simonw 2y agoYeah I'm very impressed that the tiniest quant version appears partially usable... especially now I've replicated that same S, T, R, A, W, B, E, R, F, U, R, Y bug on a much larger one: https://news.ycombinator.com/item?id=43018494 https://news.ycombinator.com/item?id=43018494