4 ms·
Read the pdf https://arxiv.org/pdf/2402.17764.pdf https://arxiv.org/pdf/2402.17764.pdf they call it 1-bit everywhere. I don't know why do they do this, 1-bit s
by bmacho 3y ago
Read the pdf https://arxiv.org/pdf/2402.17764.pdf https://arxiv.org/pdf/2402.17764.pdf they call it 1-bit everywhere.
I don't know why do they do this, 1-bit seems to be a very wrong name for {-1, 0, 1}.
- FrustratedMonky 3y agoYes Technically, but it is catchy for the masses. 1-bit seems to get the idea across, even if not technically describing {-1,0,1}.
- edflsafoiewq 3y agoI think 0 "doesn't count", since you don't have to add or subtract anything for it, just mask it out.
- numpad0 3y agoTernary or three-value logic is a thing in CS[1] 1: https://en.wikipedia.org/wiki/Three-valued_logic https://en.wikipedia.org/wiki/Three-valued_logic
- paipa 3y agoWould be cool to see what happens if you quantize towards zero preferentially. Sparsifying the matrix should improve inference speed directly, right?