3 ms·
I’m a layperson when it comes to this topic but does this mean every value in the network is a value from 00 to 11? I.e: 00, 01, 10, and 11? I struggle to unde
by saberience 3y ago
I’m a layperson when it comes to this topic but does this mean every value in the network is a value from 00 to 11? I.e: 00, 01, 10, and 11?
I struggle to understand how a network with only two bits of precision could ever generate text or numbers or anything really.
Is my intuition wrong here? If so, can someone give an example of what it means to quantize the network down to 2 bits only.
- p1esk 3y agoIt’s better to think about mapping groups of weights to an entry in a large codebook, and then using clever methods to do weight lookups. 2 bits of precision per weight is perfectly fine as long as you have enough weights. The information encoded by a neural network is measured in total number of bits, so you can compress it by either reducing number of weights or reducing number of bits per weight.