Tensors cores were built for training. For inference they added the int8 instructions (dp4a) which have lower precision. The Turing also has int4, and for infer

Tensors cores were built for training. For inference they added the int8 instructions (dp4a) which have lower precision. The Turing also has int4, and for inference this card blows the Volta out of the water since the v100 was only ~TOPS for int8. The Turing is 250 TOPS for int8 and 500 TOPS for int4.