> what advantage do I get from the compute cost of gradient descent over random sampling? Random sampling becomes prohibitive in higher dimensions due to the c

> what advantage do I get from the compute cost of gradient descent over random sampling? Random sampling becomes prohibitive in higher dimensions due to the curse of dimensionality [0]. Gradient descent doesn't have this problem and will always converge to a local (but, as can be seen here, not necessarily an absolute) minimum. The step size effectively controls how far from the "real" local minimum you can get: too big a step size and you end up repeatedly "jumping over" the minimum. [0] - https://en.wikipedia.org/wiki/Curse_of_dimensionality https://en.wikipedia.org/wiki/Curse_of_dimensionality