3 ms·
Regularization in Neural Networks
- Buttons840 4y agoDoes anyone know if preactivation normalization is a thing? Instead of doing batch normalization and forcing the preactivations to be a normal distribution, what if we added a penalty to the loss to merely incentivize the preactivations to be normal?