Yes. I'd go further - the "nice fitting algorithm" is to minimise the error on the training set as a function of the weight parameters, and one obvious way to

Yes. I'd go further - the "nice fitting algorithm" is to minimise the error on the training set as a function of the weight parameters, and one obvious way to (locally) minimise that is gradient descent + the chain rule. The math / applied math machinery is all incredibly general, and a useful way to think about many machine learning algorithms. http://en.wikipedia.org/wiki/Gradient_descent http://en.wikipedia.org/wiki/Gradient_descent http://en.wikipedia.org/wiki/Chain_rule#Higher_dimensions http://en.wikipedia.org/wiki/Chain_rule#Higher_dimensions Not to say that applied math is the only valuable perspective to think about, there's clearly also statistical and computational views as well. E.g. we're trying to approximate a function that we can never directly evaluate (error on the samples we havent seen yet)