4 ms·

Could you please forward me to a resource that explains this connection?

• punnerud 4y ago
  It was in one of the fast.ai courses, I think where Jeremy did back propagation using Excel https://www.fast.ai/ https://www.fast.ai/ Could be that someone else here remember the exact video
• deleted 4y ago
  [deleted]
• montebicyclelo 4y ago
  Hope you don't mind me plugging my blog post, that covers chain rule -> autodiff -> training of nn. https://sidsite.com/posts/autodiff/ https://sidsite.com/posts/autodiff/
  • iamcreasy 4y ago
    Absolutely not. Thank you for sharing.
• FabHK 4y ago
  The reverse mode has famously been re-discovered (or re-applied) many times, for example as backpropagation in ML, and as AAD in finance (to compute "Greeks", ie partial derivatives of the value of a product wrt many inputs). A few resources here: An overview, with a bias towards finance: https://informaconnect.com/a-brief-introduction-to-automatic-adjoint-differentiation-aad/ https://informaconnect.com/a-brief-introduction-to-automatic... On the history: Andreas Griewank, Who Invented the Reverse Mode of Differentiation? https://ftp.gwdg.de/pub/misc/EMIS/journals/DMJDMV/vol-ismp/52_griewank-andreas-b.pdf https://ftp.gwdg.de/pub/misc/EMIS/journals/DMJDMV/vol-ismp/5... On the history of back propagation: https://en.wikipedia.org/wiki/Backpropagation#History https://en.wikipedia.org/wiki/Backpropagation#History The article that introduced it to finance: Michael Giles and Paul Glasserman, Smoking adjoints: fast Monte Carlo Greeks https://www0.gsb.columbia.edu/faculty/pglasserman/Other/RiskJan2006.pdf https://www0.gsb.columbia.edu/faculty/pglasserman/Other/Risk... Survey of the application in finance: Cristian Homescu, Adjoints and Automatic (Algorithmic) Differentiation in Computational Finance https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1828503 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1828503