3 ms·
Thank you. You compared each model fixed number of times (images/sec). What was the training accuracy difference between FP32 and FP16? Do they match? I woul
by nabla9 8y ago
Thank you.
You compared each model fixed number of times (images/sec). What was the training accuracy difference between FP32 and FP16? Do they match?
I would like find out the efficiency gain when training he same model to same accuracy with different floating points types and the same model size.
- sabalaba 8y agoThe training accuracy won't match. Here's an excerpt from a presentation on FP16 training to give you an idea of the loss in accuracy: Mode | Top1 accuracy, % | Top5 accuracy, % -------------------------+------------------+--------------------- Fp32 | 58.62 | 81.25 Mixed precision training | 58.12 | 80.71 FP16 training | 54.89 | 78.12 [1] http://on-demand.gputechconf.com/gtc/2017/presentation/s7218-training-with-mixed-precision-boris-ginsburg.pdf http://on-demand.gputechconf.com/gtc/2017/presentation/s7218...