3 ms·
There is a random matrix theory derived diagnostic of training that relies on the spectral density of the correlation matrix of the weights. Each layer's spectr
by esafak 3y ago
There is a random matrix theory derived diagnostic of training that relies on the spectral density of the correlation matrix of the weights. Each layer's spectral density is fit to a truncated power law, and deemed properly trained if the power law exponent alpha is just above two.
https://jmlr.org/beta/papers/v22/20-410.html https://jmlr.org/beta/papers/v22/20-410.html