4 ms·
Spectral Normalization Explained
- visarga 9y agoPlease can anyone who understood the math explain in simpler terms how eigenvalues are related to gradient norm?
- mlevental 9y agoit's right there in the proof; it's just a convenient way of working out the Lipschitz constant for a linear function (i.e. matrix * vector) and the Lipschitz constant bounds the gradient.
- gooselabs 9y agoThe spectral norm is really just another word for the largest singular value of a matrix. And the singular values of a matrix A are the square root of the eigenvalues of A transpose times A. (If A is a square matrix, then the singular values are just the absolute value of the eigenvalues.)