> Gradient descent is similar to a ball with no mass falling down a hill. Other methods (heavy-ball and Nesterov acceleration) are similar to a heavier ball rol

> Gradient descent is similar to a ball with no mass falling down a hill. Other methods (heavy-ball and Nesterov acceleration) are similar to a heavier ball rolling down a hill. I've written a post [0] on this with a visualization. I don't think mass is the right analogy -- if you non-dimensionalize a ball on a hill the mass shouldn't even appear. Maybe critical vs under damped (gradient descent isn't allowed to overshoot, Nestersov methods are)? Edit: OK, with a viscous drag force the analogy sort of works, a heavier sliding particle will overshoot. A massless particle will not move at all, however...