FYI, training - make model out of training dataset and a predefined architecture. Inference - use model for predictions/classifications. Usually training runs i

FYI, training - make model out of training dataset and a predefined architecture. Inference - use model for predictions/classifications. Usually training runs in FP32 or mixed-mode FP16/FP32, inference can be often used in FP8/FP16 after model pruning and normalization. Nothing to do with probabilistic inference, PGM etc. All it tells you there are some tensor cores running fast with low precision and some are now super popular like Tesla T4. For Bayesian inference you might be forced to use super expensive FP64 cards instead, as logarithm sums and small multiplications can get ugly pretty fast in FP32.