9 ms·
The author points out that Newton's method needs a good initial guess. This is especially true if you have a function that is not well approximated by a quadrat
by montecarl 7y ago
The author points out that Newton's method needs a good initial guess. This is especially true if you have a function that is not well approximated by a quadratic polynomial. If, for example, you want your optimization to converge to a nearby minima (and no explode off towards infinity) a very simple fix is to implement a maximum step size that limits how far the next x value is from the current one.
This technique is used when optimizing atomic structures, where the energy changes rapidly as a function of distance between the atoms. Newton's method (or more typically, one of its approximations, like L-BFGS) without a maximum step size would just blast all the atoms apart, but with a maximum step size, it works extremely well.
This technique does require some domain knowledge. That is, what step size is "big", which is not always easy to know.