It's pretty clear from the article that 8-bit inference is referring to FP8, as the author claims int8 is similar to fp16 inference performance. > Ada/Hopper a

It's pretty clear from the article that 8-bit inference is referring to FP8, as the author claims int8 is similar to fp16 inference performance. > Ada/Hopper also have FP8 support, which makes in particular 8-bit training much more effective. I did not model numbers for 8-bit training because to model that I need to know the latency of L1 and L2 caches on Hopper/Ada GPUs, and they are unknown and I do not have access to such GPUs. On Hopper/Ada, 8-bit training performance can well be 3-4x of 16-bit training performance if the caches are as fast as rumored. For old GPUs, Int8 inference performance for old GPUs is close to the 16-bit inference performance.