I took a machine learning class in ~2006 and granted the prof wasn't specializing in studying NNs, but he seemed to have little awareness of the impact of datas

I took a machine learning class in ~2006 and granted the prof wasn't specializing in studying NNs, but he seemed to have little awareness of the impact of dataset size, convnets were never mentioned, the suggested approach was two-layer MLPs and sigmoid activation functions. The course was based on the book "Artificial Intelligence: A Modern Approach" 2nd edition, which barely just mentions neural networks in passing. I think there just weren't enough people looking at neural networks back then. At the time, they were still considered a niche machine learning technique, just one tool in the toolbox, and not necessarily the best one. Also, it's one thing to say that the best of the best had access to computation, but if your university has just one supercomputer, which is shared among everyone, and you have a limited number of hours during which you can use it, that's not increasing your chances at running very thorough experiments. If anything, it's not that we didn't have the compute to do deep learning in the 90s and early 2000s, it's that more access to compute makes a broader range of experimentation a lot more accessible. That makes it a lot easier to find the right training recipe more quickly. Back in 2009 the enthusiasm around neural nets was starting to grow, as there were some early successes with deep neural nets, but I remember some people were still trying to argue that support vector machines were just as effective for image classification. It seemed laughable to me at the time, but I think that everyone who wasn't a connectionist was starting to feel threatened and wanted to justify their chosen research specialty, which they would keep doing until they couldn't anymore.