4 ms·
NVIDIA says: 44 TOPS INT8 (new deep learning inferencing instruction) I think it's a related to storing float arrays as arrays of 8-bit integers in memory and
by nabla9 10y ago
NVIDIA says: 44 TOPS INT8 (new deep learning inferencing instruction)
I think it's a related to storing float arrays as arrays of 8-bit integers in memory and converting them into floats just before using. It's 2x more space efficient than fp16
http://www.kanadas.com/program-e/2015/11/method_for_packing_int8_arrays.html http://www.kanadas.com/program-e/2015/11/method_for_packing_...
- astrodust 10y agoYou can get away with 8-bit floats on many neural network type applications so that might be the idea here.
- bitfiddler0 10y agoIt's an 8-bit int type though. Perhaps even that's sufficient for inference?
- astrodust 10y agoFor a lot of network types that will do the job. 2x the performance is often better than 2x the fidelity. You can get higher accuracy with more nodes.