I think for the application here (analogous to autograd on neural networks) you know each individual building block of the function (a well-defined neural netwo

I think for the application here (analogous to autograd on neural networks) you know each individual building block of the function (a well-defined neural network layer) so can be sure such things will not happen.