Gradient descent seeks the lowest prediction error when compared with the training answer. A 1D line generally has an easy gradient downhill. (x is input, y is

Gradient descent seeks the lowest prediction error when compared with the training answer. A 1D line generally has an easy gradient downhill. (x is input, y is error). Each input is a dimension. In 2, 3, or more dimensions, downhill can be subtle. Calculus is used to find the slope at a multidimensional point. You descend in the direction the calculus proposes, which is downhill toward the lower error. The error is a scalar, it is chopped up proportionally by size of contribution. Each input is weighted, the weights are adjusted downward. You are trying to minimize the error in input space. To attribute blame for the error so it can be minimized, the calculus chain rule passes the buck back to the inputs - this is back propagation. Multilayer networks do this multiple times: pass the error back, chopped up according to which input made the mistake (proportionally). So the end result is a series of adjustments to all the neural weights which will reduce this error. Rinse and repeat many many times and you will get close to a generalised solution. Local minima used to be a worry; but in practice the more dimensions the less local minima differ from the global error minima.