Just a quick question/remark: I have a feeling that it's best to think about DNNs as approximating a function, not a program. (only then you obtain a program as

Just a quick question/remark: I have a feeling that it's best to think about DNNs as approximating a function, not a program. (only then you obtain a program as a result of applying this function). But because mathematically, you can formalise your big NN as one big parameterised function, I think it's more correct to view a NN as a function... Optimisation in program space would be trying to find both the structure (connections and activation functions) and weights of the NN, which is not what we do currently. We tend to hand/engineer (or keep what works best empirically) the architecture (1), then train by finding the best weights. I am very interested in approaches to efficiently opmitise in program space, and DL/backprop doesn't feel like it (1) although this is starting to change.