3 ms·
These results seem important for nonconvex optimization in general, but for ML applications where we usually use stochastic/batch gradient descent, I wonder if
by Eridrus 9y ago
These results seem important for nonconvex optimization in general, but for ML applications where we usually use stochastic/batch gradient descent, I wonder if the the stochasticity adds enough perturbation for this to not really be that useful.
- Cacti 9y agoWell, in general, no, there is no perturbation method large (or good) enough to get out of saddle points, including via stochastic gradient descent. It might work in a particularly specific problem, but not in general.
- Eridrus 9y agoFine, but I mostly care about ML applications, where I'm wondering if this is expected to help at all.