4 ms·

There was a revolution, when they started using backpropogation to optimize the gradient search. It's also why I don't agree with calling them "neural" anythin

• eliben 9y ago
  Backpropagation was first applied to Neural Networks over 30 years ago [Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (8 October 1986). "Learning representations by back-propagating errors". Nature]
  • nightski 9y ago
    Not to deep nets iirc, which was one of the advancements.
    • pedrosorio 9y ago
      The GP implies that was not tried before: "There was a revolution, when they started using backpropogation to optimize the gradient search". "Back-propagation allowed researchers to train supervised deep artificial neural networks from scratch, initially with little success. Hochreiter's diploma thesis of 1991[1][2] formally identified the reason for this failure in the "vanishing gradient problem", which not only affects many-layered feedforward networks,[3] but also recurrent networks." https://en.m.wikipedia.org/wiki/Vanishing_gradient_problem https://en.m.wikipedia.org/wiki/Vanishing_gradient_problem