3 ms·
The accuracy loss is more consistent with some kind of quantization of the model(-s) behind the scenes than the alignment gone wrong. Quantization to serve more
by practice9 3y ago
The accuracy loss is more consistent with some kind of quantization of the model(-s) behind the scenes than the alignment gone wrong. Quantization to serve more users faster, on same amount or less of compute.
- arrowsmith 3y agoSorry, what does quantization mean here?
- imdsm 3y agoReducing the precision of the parameters — result being less memory intensive
- iamjackg 3y agoReducing the precision of the weights from high precision floating points to either lower precision floats or even integers. You'd think it would greatly reduce the performance of a model, but in most cases the decline in quality is extremely tolerable compared to the reduction in memory/processing requirements.
- mlboss 3y agoIt means using less number of bits to store float values. This reduces the memory/compute requirement at the cost of making model less precise.