4 ms·
You can map the weights to an uncompressed (Q,A) space, and then map them back again to weight-space -- all without loss of information. The actual domain space
by mjburgess 3y ago
You can map the weights to an uncompressed (Q,A) space, and then map them back again to weight-space -- all without loss of information. The actual domain space they were compressed from is irrelevant; their values are equivalent.
- MacsHeadroom 3y agoQ & A are not part of pretraining.