3 ms·
Inference is often done fixed point, whereas training is (usually) floating point. Inference also prefers different IO patterns, because you don't need to keep
by londons_explore 5y ago
Inference is often done fixed point, whereas training is (usually) floating point.
Inference also prefers different IO patterns, because you don't need to keep the activations for every layer ready for backpropogation.