3 ms·
Properly tuned (DistributedDataParallel + mixed precision) it will train faster, and consume a lot less RAM, allowing you to use larger batches, and higher LR.
by m0zg 6y ago
Properly tuned (DistributedDataParallel + mixed precision) it will train faster, and consume a lot less RAM, allowing you to use larger batches, and higher LR.