Instead of bigger and badder networks with zillions of layers, can we use the opposite approach: reducing the network to minimal size at which it works and rese

Instead of bigger and badder networks with zillions of layers, can we use the opposite approach: reducing the network to minimal size at which it works and researching why it works(Reduced case -> General rule). It will be much simpler to make current network easier to analyze, than trying to find relations in a giant multilayer soup. Perhaps even creating a neural network optimizer to minify the target network, increasing its efficiency without compromising the power at the tasks it built for.