It's been a while since I've watched this talk[0], but iirc the upshot is that if you think of the "differentiation at p" operator as sending f to the pair (f(p

It's been a while since I've watched this talk[0], but iirc the upshot is that if you think of the "differentiation at p" operator as sending f to the pair (f(p), D_p f) instead of the usual D_p f, then you get a nice compositional operator that defines a functor on vector spaces, automatic differentiation more or less pops out, and you get reverse mode by looking at what this does when applied to dual spaces. So in some sense it's maybe not hard, though any attempt to do it without building up some theory (e.g. abstract vector spaces and dual spaces) first will probably come across as magic. On the other hand, magic tricks would be right at home in an intro differential equations class so maybe this would be a perfect addition. Or it can replace Wronskians or something. [0] https://youtube.com/watch?v=17gfCTnw6uE https://youtube.com/watch?v=17gfCTnw6uE