I feel differentiable programming is not really there. I understand the specific case of numerical functions, it may make sense if you just want to tweak some

I feel differentiable programming is not really there. I understand the specific case of numerical functions, it may make sense if you just want to tweak some parameters a bit like a nn, or for example in quantum diff programming people seem to tweak parameters in some Hamiltonian. What I actually want is something you will actually change the structure of the program somehow. If you pretend the entire program is a function f, parameters are just points on your program. Why is moving to different points differentiating. Differentiating is producing a new function f’. In the case of the NN it makes sense to select different points. But then we do also modify the structure with dropouts etc.