3 ms·
> This represents an almost 8x compression ratio for every weight matrix in the transformer model Surely you’d need more ternary weights though to achieve same
by Havoc 2y ago
> This represents an almost 8x compression ratio for every weight matrix in the transformer model
Surely you’d need more ternary weights though to achieve same performance outcome?
A bit like a Q4 quant is smaller than a Q8 but also tangibly worse so the “compression” isn’t really like for like
Either way excited about more tenary progress.
- areddyyt 2y agoWe do quantization-aware training, so the model should minimize the loss w.r.t. the ternary weights, hence no degradation in performance.