> The thing is, AD predated CT substantially, and was well-understood for a long time. The original theoretical description of AD is something I understood, and

> The thing is, AD predated CT substantially, and was well-understood for a long time. The original theoretical description of AD is something I understood, and the CT description of it was impenetrable, and not useful. I was the one who linked that. I wasn't familiar with AD before his talk, but I find it perfectly understandable (modulo Haskell's sometimes foreign-to-me syntax). I think the relevant background here is that I was a math major and had a course in abstract linear algebra (i.e. general vector spaces over fields) and two semesters of abstract algebra (groups, rings, fields) plus dipped my toes in some CT on my own. He basically notes that D had a slightly nicer chain rule if you write D'f(a)= (f(a), Df(a)) and use D' instead. Then his definition of AD is just the chain rule and the product rule, plus the fact that Df = f for all linear f, applied recursively. So forward mode AD is just writing down facts you learned in calc 1 as a recursive definition (with, I guess, some base cases like sin/cos hard coded). His description of reverse mode AD is more complicated, but IMO still accessible to someone who's seen dual spaces, product/coproduct vector spaces and maybe enough CT to understand that the map from a vector space to its dual (and matrices to their adjoint) is a contravariant functor. The basic observation is that he can write the chain rule and product rule as a compositions of linear maps with product/coproduct spaces (which are the same thing for vector spaces). Then he says a biproduct category is one that has (equal) product/coproduct, and notes that the dual map preserves that structure. The point is that biproducts are what he needs to write chain/product rules as compositions of linear maps. Since he wrote D' recursively in terms of linear maps, and dualizing/taking the adjoint still gives him the structure he needs to do that, he's able to write reverse mode AD as the adjoint representation of D'. He just needs to write the adjoints of the pieces he used, and again compute recursively. I know at least at my school, the engineering program I was in had a first semester graduate class that dealt some with dual representations (using "test functions" etc. when solving differential equations), so I think it could be distilled down to something appropriate for masters level engineering while retaining the character of what he presented. Like I said, the Haskell syntax was probably the most obscure thing for me. It's all a matter of perspective and background, but IMO it's sort of like how if you're familiar with linear algebra (including function spaces), then the Fourier transform is just writing a function as a sum of projections onto an orthonormal basis. No need to remember the formulas for it; they're obvious and have simple geometric meaning.