4 ms·
The article, as usual, is terrible. I think this paper is more about figuring out the optimal step sizes.
by cbsmith 3y ago
The article, as usual, is terrible. I think this paper is more about figuring out the optimal step sizes.
- wenc 3y agoTypically optimal step sizes (α) along a descent direction in gradient descent are determined using a line search, which is itself a (cheaper) univariate optimization problem. https://en.wikipedia.org/wiki/Line_search https://en.wikipedia.org/wiki/Line_search Most nonlinear optimizers already do this.