4 ms·
Does anyone know if preactivation normalization is a thing? Instead of doing batch normalization and forcing the preactivations to be a normal distribution, wh
by Buttons840 4y ago
Does anyone know if preactivation normalization is a thing?
Instead of doing batch normalization and forcing the preactivations to be a normal distribution, what if we added a penalty to the loss to merely incentivize the preactivations to be normal?