4 ms·
µTransfer A technique for hyperparameter tuning of enormous neural networks
- homarp 5y agotwitter summary: https://mobile.twitter.com/TheGregYang/status/1501294412126560257 https://mobile.twitter.com/TheGregYang/status/15012944121265... "You can't train GPT-3 on a single GPU, much less tune its hyperparameters (HPs). But what if I tell you… …you can tune its HPs on a single GPU thanks to new theoretical advances?"