12 ms·Inference usually runs on much more quantized models, which don’t use bfloat16.by adastra22 3y agoInference usually runs on much more quantized models, which don’t use bfloat16.