3 ms·
ideally if ternary models work, the math is extremely easy for computers (addition/subtraction vs 16 bit multiplication)
by c0rruptbytes 4mo ago
ideally if ternary models work, the math is extremely easy for computers (addition/subtraction vs 16 bit multiplication)
- jjcm 4mo agoNot quite as I understand it. The ternary approach bonsai uses leverages a FP16 scaling factor that each value in the ternary maps to. You're still using 16 bit multiplication, it's just that the weights are far more compressed.
- c0rruptbytes 4mo agofair, i think i was referring more to 1.58 bit architecture in general since the original paper (Figure 3) shows that we eliminate FP16 multiplication and addition just for INT8 addition. I need to dive deeper into bonsai overall if it differs https://arxiv.org/pdf/2402.17764 https://arxiv.org/pdf/2402.17764