Nice comment. In regard to your reference to reducing precision to FP16 for performance gains, you might want to read a recently published paper by Baidu Resear

Nice comment. In regard to your reference to reducing precision to FP16 for performance gains, you might want to read a recently published paper by Baidu Research and NVIDIA teams on mixed precision training of deep learning models (link to the paper is at the end of the following relevant post): https://www.nextplatform.com/2017/10/11/baidu-sheds-precision-without-paying-deep-learning-accuracy-cost https://www.nextplatform.com/2017/10/11/baidu-sheds-precisio.... Enjoy! :-)