3 ms·

It's just an application of the chain rule. It's not interesting to ask who invented it.

• qarl 1y ago
  From the article: Some ask: "Isn't backpropagation just the chain rule of Leibniz (1676) [LEI07-10] & L'Hopital (1696)?" No, it is the efficient way of applying the chain rule to big networks with differentiable nodes—see Sec. XII of [T22][DLH]). (There are also many inefficient ways of doing this.) It was not published until 1970 [BP1].
  • uoaei 1y ago
    The article says that but it's overcomplicating to the point of being actually wrong. You could, I suppose, argue that the big innovation is the application of vectorization to the chain rule (by virtue of the matmul-based architecture of your usual feedforward network) which is a true combination of two mathematical technologies. But it feels like this and indeed most "innovations" in ML is only considered as such due to brainrot derived from trying to take maximal credit for minimal work (i.e., IP).
    • qarl 1y ago
      The real metric is whether anyone remembers it in 100 years. Any other discussion just comes off as petty.
• piperly 1y ago
  You got it right: Leibniz!