4 ms·
I've read a lot about fp16 being good enough for training, but the thing that people don't mention is that just swapping fp32 for fp16 will make things fail bec
by Eridrus 10y ago
I've read a lot about fp16 being good enough for training, but the thing that people don't mention is that just swapping fp32 for fp16 will make things fail because your deep learning framework doesn't implement everything you use for fp16, and after you fix that, it will probably diverge because standard practices aren't used to dealing with such a limited range.
Which isn't to say that the Deep Learning stacks won't get there eventually, but at the moment it's not as easy as flipping a switch.
- aab0 10y agoFor a 4x boost, I imagine Tensorflow and Torch will get support shortly after the GPUs start shipping in real quantities.
- Eridrus 10y agoIt's a 2x boost for training since you can't use int8 for training and need fp16. INT8 is a 4x for inference, but most people aren't using GPUs for inference atm.
- scottlegrand 10y agoSome of us are... https://blogs.aws.amazon.com/bigdata/post/TxGEL8IJ0CAXTK/Generating-Recommendations-at-Amazon-Scale-with-Apache-Spark-and-Amazon-DSSTNE https://blogs.aws.amazon.com/bigdata/post/TxGEL8IJ0CAXTK/Gen...
- Eridrus 10y agoFair enough; when I said most people I meant companies who are not AmaGooFaceSoft and their international equivalents. Though I can't quite tell if you're doing GPU batch predictions and storing them or doing them in realtime with Spark Streaming. Unrelated question though: any chance you will do blog post/paper about how DSSTNE does automatic model parallelism and gets good sparse performance compared to cuSparse/etc?