Probably current gen GPUs can bridge the gap. The nvidia platform after Drive PX 2 (which is what Tesla ships) uses Volta (with "tensor cores" to speed up fp16)

Probably current gen GPUs can bridge the gap. The nvidia platform after Drive PX 2 (which is what Tesla ships) uses Volta (with "tensor cores" to speed up fp16) and has a deep learning accelerator (http://nvdla.org http://nvdla.org) for inference (i think both int8 and fp16).