3 ms·
In what domains do people do 8 bit machine learning? Most example code I've seen e.g. for TensorFlow uses float32 matrices.
by radarsat1 7y ago
In what domains do people do 8 bit machine learning? Most example code I've seen e.g. for TensorFlow uses float32 matrices.
- jdietrich 7y agoIt's quite common to train at float32 but quantize down to int8 for inference. The loss of accuracy is negligible, but you make huge performance gains. https://medium.com/apache-mxnet/model-quantization-for-production-level-neural-network-inference-f54462ebba05 https://medium.com/apache-mxnet/model-quantization-for-produ...