3 ms·
I've seen this before; it was called "over-relaxing the step", I think. The idea is that even if a step is too big, we can still show that the step was in the r
by lidHanteyk 6y ago
I've seen this before; it was called "over-relaxing the step", I think. The idea is that even if a step is too big, we can still show that the step was in the right direction. That's the same as the author's technique of taking multiple normal-sized steps in a single direction.
I don't know the name for the other part of the author's technique, where several Newton-Raphson steps are taken, but I've seen it before, too. Crucially, the author takes ratios of first and second derivatives, rather than of the original function and first derivative, which is the right thing to do for minimization but is going to be perhaps more expensive than the original gradient computation.