My feeling is that since shallow networks can be made to have equivalent accuracy to deep networks, that the real challenge isn't topology but training. In a s

My feeling is that since shallow networks can be made to have equivalent accuracy to deep networks, that the real challenge isn't topology but training. In a sense, topology is training. One reason that DNNs outperform shallow networks for so many problems is that presumably their topology captures structure inherent in the world that would otherwise have to be discovered by a simpler network.