5 ms·
Is there a relationship between the power method and some "standard" optimization algorithm (grad. descent, Newton's, ...) applied to maximization of the Raylei
by chestervonwinch 9y ago
Is there a relationship between the power method and some "standard" optimization algorithm (grad. descent, Newton's, ...) applied to maximization of the Rayleigh quotient?
- nimish 9y agoConjugate gradient (And other krylov subspace iterations) are basically a generalization of power iterations where you consider the whole set of {b, Ab, A^2b...} rather than just the one.
- selimthegrim 9y agoMATLAB, for one, in its eigs function uses Arnoldi with restart (picking a arbitrary seed to zoom on "good" parts of the Krylov subspace). Imagine we pick an Arnoldi vector for restarting (a member of that orthonormal basis which spans that subspace) which is equivalent to some polynomial expression of the initial matrix times our initial vector. We essentially want to pick this vector using such a polynomial (not the characteristic) so that the values of this polynomial function peak as a function of the true eigenvalues of the matrix - this lets us pick out the Krylov space that better approximates the correct eigenvector and hence the correct eigenvalue.
- hyperbovine 9y agoThe leading eigenspace is the best rank one approximation of A. So the power method finds a minimizer v of the (Frobenius) norm of A-vv^T.