I tried training a 4 layer network in as a young man (because, why not) a day later, no convergence, some calculations revealed that an expectation of many 1000

I tried training a 4 layer network in as a young man (because, why not) a day later, no convergence, some calculations revealed that an expectation of many 1000's of days (I have a memory of 1 million, but that could be bs) before the network would converge. So "deep" networks have been around for many decades, and they haven't, because you couldn't train them. Now we have computers that are 10,000* faster (at least) and training algorithms that are much faster too these architectures are interesting.