4 ms·
Well, I too put it on my "to read" stack but if it is referring to ordinary automatic differentiation, also known as "algorithmic differentiation" that beast is
by joe_the_user 10y ago
Well, I too put it on my "to read" stack but if it is referring to ordinary automatic differentiation, also known as "algorithmic differentiation" that beast is a bit more specific.
That is, ad allows the differentiation of functions defined as algorithms but not with arbitrary "branching and control flow" but essentially only when the "algorithm" involve a composition of primitive functions having known derivatives (example: addition, multiplication, exponentiation). That is still a big set - back propagation uses one form of AD for example.
However, for example, AD cannot be correctly applied to functions defined with loop or with arbitrary recursion.
I'm not sure if there's a relation to the DeepMind DNC but maybe if I read article I'll find out.
See:
https://en.wikipedia.org/wiki/Automatic_differentiation https://en.wikipedia.org/wiki/Automatic_differentiation