> if we scale the activations in a particular layer in a non-linear network, some neurons in later layers may ‘activate’ or ‘deactivate’. Normalization remove

> if we scale the activations in a particular layer in a non-linear network, some neurons in later layers may ‘activate’ or ‘deactivate’. Normalization removes this problem. Magnitude information can still be encoded separately in a log form so differentiation can still happen when scale matters, but scaling doesn't have much impact by default (small initial weights following magnitude element).