3 ms·
Yannic Kilcher has a great video on this out already, including some thoughtful critique. https://youtu.be/rNkHjZtH0RQ https://youtu.be/rNkHjZtH0RQ One of the
by Yenrabbit 6y ago
Yannic Kilcher has a great video on this out already, including some thoughtful critique. https://youtu.be/rNkHjZtH0RQ https://youtu.be/rNkHjZtH0RQ
One of the things he mentioned was that they introduce this fancy alternative to BatchNorm AND come up with a fancy new architecture. The combination does really well, but it isn't clear how much of the gain is due to the new improved architecture Vs the 'adaptive gradient clipping' they introduce.