4 ms·
Is this useful for training as well as running a model. Or is this approach specifically for running an already-trained model faster?
by jonplackett 3y ago
Is this useful for training as well as running a model. Or is this approach specifically for running an already-trained model faster?
- frozenport 3y agoIn principle, training is basically the same as running inference but iteratively, in practice training would use a different software stack.
- robrenaud 3y agoTraining requires a lot more memory to keep gradients + gradient stats for the optimizer, and needs higher precision weights for the optimization. It's also much more parallelizable. But inference is kind of a subroutine of training.
- tome 3y agoCurrently graphics processors work well for training. Language processors (LPUs) excel at inference.