The we are learning is seems like a natural extension of multi-variable calculus and PDEs. As if what happens when you have hundreds of variables ? What tools d

The we are learning is seems like a natural extension of multi-variable calculus and PDEs. As if what happens when you have hundreds of variables ? What tools do we turn towards ? The thing is for example in ML the standard solution is to use gradient decent, but what I learnt is depends highly on the type of equations you are dealing with. The first tool should never be a hammer. We used stochastic gradient decent on linear machine tasks ! Now after learning optimization theory it seems like madness. The thing is whenever you introduce unnecessary complexity you pay technical debt. Occam's razor is really important when you are dealing with modelling.