I'd say this is more of a result of decades of trial and error in different architectures/topologies rather than some fundamental property of neural networks in

I'd say this is more of a result of decades of trial and error in different architectures/topologies rather than some fundamental property of neural networks in general.