Say what? You can get insane single-precision efficiency out of GTX 980. Double-precision OTOH is a disaster (1/150 yes 1/150 as opposed to GK104's 1/30 or so

Say what? You can get insane single-precision efficiency out of GTX 980. Double-precision OTOH is a disaster (1/150 yes 1/150 as opposed to GK104's 1/30 or so). But for most machine learning, double-precision is overkill. IMO What remains to be seen is what one could do with FP16 data mixed with FP32 accumulation. My signal processing friends think a pure FP16 network is just asking for trouble and are of the opinion that a minimum of 20 bits is needed for stable accumulation and 24 is more than enough. That's not far off from Teradeep's 8-bit data plus 16-bit accumulators for image data.