5 ms·You're running fp32 models, fp16 or quantized?by phh 3y agoYou're running fp32 models, fp16 or quantized?tome 3y agoFP16 for calculating all activations. Some data is stored as FP8 at rest.