3 ms·
Please can anyone who understood the math explain in simpler terms how eigenvalues are related to gradient norm?
by visarga 9y ago
Please can anyone who understood the math explain in simpler terms how eigenvalues are related to gradient norm?
- mlevental 9y agoit's right there in the proof; it's just a convenient way of working out the Lipschitz constant for a linear function (i.e. matrix * vector) and the Lipschitz constant bounds the gradient.
- gooselabs 9y agoThe spectral norm is really just another word for the largest singular value of a matrix. And the singular values of a matrix A are the square root of the eigenvalues of A transpose times A. (If A is a square matrix, then the singular values are just the absolute value of the eigenvalues.)