3 ms·

The search strategy/optimizer is actually gradient descent, backpropagation is just an efficient way to compute gradients.

• imaltont 4y ago
  Both Tom M. Mitchell's "Machine Learning" as well as Russel & Norvig's "Artificial Intelligence: A Modern Approach" define the whole process from propagating the input until you have an output, calculate the gradient and update the weights.
  • tehsauce 4y ago
    [4] Goodfellow, Bengio & Courville 2016, p. 200, "The term back-propagation is often misunderstood as meaning the whole learning algorithm for multilayer neural networks. Backpropagation refers only to the method for computing the gradient, while other algorithms, such as stochastic gradient descent, is used to perform learning using this gradient."
    • imaltont 4y ago
      As good as Ian Goodfellow's work has been, I think I have to disagree with him on what is part of the back propagation algorithm. Rumelhart et. al 1986 also describe it is a single algorithm from start to end. It is true you can use other things than stochastic gradient descent though, Mitchell also points this out by saying their algorithm was a backpropagation example using gradient descent.