2 ms·FP16 for calculating all activations. Some data is stored as FP8 at rest.by tome 3y agoFP16 for calculating all activations. Some data is stored as FP8 at rest.