3 ms·
I don't think it's quite so clear cut, and the papers and names are often a bit more precise anyway. BitNet is (I think) actually one bit per parameter. BitNet
by IanCal 2y ago
I don't think it's quite so clear cut, and the papers and names are often a bit more precise anyway.
BitNet is (I think) actually one bit per parameter. BitNet 1.58b isn't, but then to be fair isn't describing itself as a 1 bit llm. I'm less sure but it seems OneBit is 1 bit. One of the processes mentioned in the article is a mix of one and two bits for the weights.