While not necessarily in a first course, it is an important difference to appreciate which saves a lot of headache later. Eg: Vanilla gradient descent involves

While not necessarily in a first course, it is an important difference to appreciate which saves a lot of headache later. Eg: Vanilla gradient descent involves naively adding a covariant and a contravariant quantity, which is generically a recipe for disaster, and nicely motivates second order methods that use the Hessian as the "metric" to raise/lower indices. The picture about Hessian eigenvalues determining the step-size is of course complementary and helps gain an intuition for the meaning of a "metric" and its importance.