3 ms·
> To avoid getting stuck in local minima, we make sure we use the proper learning rate This sentence is extremely misleading. It implies that gradient descent
by blt 8y ago
> To avoid getting stuck in local minima, we make sure we use the proper learning rate
This sentence is extremely misleading. It implies that gradient descent always reaches a global minimum if the "proper" learning rate is used. Guarantees of that form are only available for restricted families of function such as Lipschitz-smooth convex functions.
- rq1 8y agoAnd you don’t necessarily want a global minimum of an universal approximator...