2 ms·
> Automatic differentiation is an elegant technique for converting a computable function expressed as a program into a derivative-computing program with similar
by carapace 7y ago
> Automatic differentiation is an elegant technique for converting a computable function expressed as a program into a derivative-computing program with similar time complexity. It does not execute the original program as a black-box, nor does it expand the program into a mathematical formula, both of which would be counter-productive. By generalizing this technique, you can produce efficient algorithms for constraint satisfaction, optimization, and Bayesian inference on models specified as programs. This approach can be broadly described as compiling into a message-passing program.
https://www.microsoft.com/en-us/research/video/from-automatic-differentiation-to-message-passing/ https://www.microsoft.com/en-us/research/video/from-automati...
- gnulinux 7y ago> nor does it expand the program into a mathematical formula. Meh, this is not entirely accurate. Sure it doesn't expand the program into analytic functions whose derivatives are easy to compute; but it still handles the program symbolically. So, in a way, it still transforms the program into a known set of mathematical primitives of which it can then construct a program that computes the derivative in compile time.
- carapace 7y agoIt seems that way to me too, but without more details about implementation I'm not sure.