4 ms·
In this paper, we find 99.9% of the gradient exchange in distributed SGD is redundant, and propose DGC to greatly reduce the communication bandwidth.
by stablemap 9y ago
In this paper, we find 99.9% of the gradient exchange in distributed SGD is redundant, and propose DGC to greatly reduce the communication bandwidth.