This is a great article. I have had to deal with this lately, and FP8 / INT8 is very important especially for mobile/on-device inference. AI will start to get

This is a great article. I have had to deal with this lately, and FP8 / INT8 is very important especially for mobile/on-device inference. AI will start to get embedded in all kind of devices (even low power one), and often backend inference is not desired or feasible. It is actually pretty awesome if the industry moves towards INT8 / FP8 inference as a standard. "But if FP8 becomes real, and if the popular training tools begin to develop ML models with FP8 as the native format, it could be a huge boon to embedded inference deployments. Eight-bit weights take the same storage space, whether they are INT8 or FP8."