That gradient descent works so well for the non-convex loss landscapes involved is quite surprising and intriguing. There are a few OK handwavy explanations; su

That gradient descent works so well for the non-convex loss landscapes involved is quite surprising and intriguing. There are a few OK handwavy explanations; supposedly most local minima of high-dimensional losses are saddle points, perhaps 'good models' have high entropy. But good theories are still far and few between