It's quite common to train at float32 but quantize down to int8 for inference. The loss of accuracy is negligible, but you make huge performance gains. https:/

It's quite common to train at float32 but quantize down to int8 for inference. The loss of accuracy is negligible, but you make huge performance gains. https://medium.com/apache-mxnet/model-quantization-for-production-level-neural-network-inference-f54462ebba05 https://medium.com/apache-mxnet/model-quantization-for-produ...