4 ms·
> Largely, their success is tied to how complicated the spectrum of the Hessian is, but that's a longer discussion. What does the "complexity" of a spectrum me
by patrick451 5y ago
> Largely, their success is tied to how complicated the spectrum of the Hessian is, but that's a longer discussion.
What does the "complexity" of a spectrum mean? Is this a statement about the uniqueness (or not) of the eigenvalues or something else entirely?
- kxyvr 5y agoIt has to do with whether the eigenvalues of the Hessian are well clustered or not. Generally, to get fast convergence near an optimal solution, the Newton system needs to be solved well enough. What well enough is can be difficult to determine and practically it's not calculated. However, if the spectrum of the Hessian is well clustered, then it doesn't take that many Krylov iterations to get an accurate solution. To be clear, how Krylov methods perform is an area of study that sits independent of optimization algorithms, but those results are applicable here. In fact, it's a slightly easier situation than normal because the Hessian is symmetric, so the spectrum is real and more exotic topics like pseudospectra don't need to be considered.