3 ms·
The eigenvalues of the Hessian of a neural network describe the curvature of the loss function. It is generally believed that in NNs trained with stochastic gr
by antognini 9y ago
The eigenvalues of the Hessian of a neural network describe the curvature of the loss function. It is generally believed that in NNs trained with stochastic gradient descent regions with high curvature tend to generalize poorly.