Typically optimal step sizes (α) along a descent direction in gradient descent are determined using a line search, which is itself a (cheaper) univariate optimi

Typically optimal step sizes (α) along a descent direction in gradient descent are determined using a line search, which is itself a (cheaper) univariate optimization problem. https://en.wikipedia.org/wiki/Line_search https://en.wikipedia.org/wiki/Line_search Most nonlinear optimizers already do this.