2 ms·
Tried similar approach several years ago but was unsatisfied with final accuracy results. Pipelining is obvious method to accelerate a sequence of computations,
by novaRom 8y ago
Tried similar approach several years ago but was unsatisfied with final accuracy results. Pipelining is obvious method to accelerate a sequence of computations, but with respect to SGD it requires some accuracy tolerance because weight updates go out of sync.
- aoeusnth1 8y agoIt seems they're not actually increasing the minibatch size here. I don't see how the updates for N microbatches of size M will be any more out of sync than a single large minibatch of size N*M.