3 ms·
Having fewer than 1 bit per weight is not absurd. E.g. you can use 2 bits to represent 3 'weights' if you insist that at most one of the weights is allowed to e
by sebzim4500 4y ago
Having fewer than 1 bit per weight is not absurd. E.g. you can use 2 bits to represent 3 'weights' if you insist that at most one of the weights is allowed to exist. If you try to order nodes so that adjacent nodes are uncorrelated the performance loss might be manageable.
People are already doing stuff like this (see sparsification) so it is conceivable to me that this is just what networks will look like in a few years.
- magicalhippo 4y ago> If you try to order nodes so that adjacent nodes are uncorrelated the performance loss might be manageable. shower thought In graphics we use barycentric coordinates to encode the position within an arbitrary triangle using two coordinates (u,v), with the third being constrained to be 1-u-v. If you order nodes to be correlated, could you use a similar trick to encode three weights for the price of two?
- hgsgm 4y agoYes, it's the same thing.