4 ms·
Previously I had thought of Levenberg Marquardt as it's own little island, separate from the collection of line search and trust region algorithms. I guess may
by xscott 5y ago
Previously I had thought of Levenberg Marquardt as it's own little island, separate from the collection of line search and trust region algorithms. I guess maybe I should think of algorithms using regularized Hessians as a third category now. (Although I guess the quasi-Newton algorithms that maintain positive definiteness are regularized in a sense, so it's kind of a fuzzy categorization)
I'm curious if this square root trick performs better than a well written trust region algorithm.