3 ms·
> We show this technique consistently outperforms validation accuracy, in some cases by up to 18.2\% compared to traditional training approaches while reducing
by eru 2y ago
> We show this technique consistently outperforms validation accuracy, in some cases by up to 18.2\% compared to traditional training approaches while reducing the computation required nearly an order of magnitude because we can now rely on smaller microbatch sizes without destabilizing training.
The accuracy improvement is great, but I'm really looking forward to the reduction in computation required!
- fchaubard 2y agoYes it will allow stable training at much smaller batch sizes. Test it out and let us know if it works for your use case!