That's my understanding, it does not compute the expression directly unless you ask it to. Could be wrong though, I've only used it a few times. AD does involv

That's my understanding, it does not compute the expression directly unless you ask it to. Could be wrong though, I've only used it a few times. AD does involve symbolic differentiation of each individual instruction, but instead of keeping the symbolic expression it is replaced with the numeric value, so the whole expression is never computed. It isn't magic and there are cases where it blows up. For instance, there are two basic ways to implement it, forward and backward propagation, each of which is good for a certain situation that the other is terrible at.