3 ms·
Exactly. The smaller bit widths from quantization might marginally decrease the compute required for each operation, but they do not reduce the overall volume o
by pkAbstract 2y ago
Exactly. The smaller bit widths from quantization might marginally decrease the compute required for each operation, but they do not reduce the overall volume of operations. So, the effect of quantization is generally more impactful on memory use than compute.
- superkuh 2y agoExcept in this case they quantized both the parameters and the activations leading to decreased compute time too.