3 ms·
You can do inference pretty easily with 8-bit fixed point weights. Now attempt doing the same during training. Training and inference are only similar at a hig
by david-gpu 9y ago
You can do inference pretty easily with 8-bit fixed point weights. Now attempt doing the same during training.
Training and inference are only similar at a high level, not in actual application.
- redcalx 9y ago... because the gradient that is being followed may have a lower magnitude than can be represented in the lower precision.
- dgacmu 9y agoYou also need a few other operations for training, such as transpose, which may or may not be fast in a particular implementation. (ETA: In case it's not obvious, I'm agreeing with david-gpu's comment, and adding more reasons that training currently differs from inference.)