3 ms·With w8a8 quantization the hw (>= hopper) can do the heavy math in fp8 twice as fast as fp16.by imjonse 2y agoWith w8a8 quantization the hw (>= hopper) can do the heavy math in fp8 twice as fast as fp16.