I disagree strongly with this view that hard problems need to be "conceptually solved" in a vacuum first. That experimentation and resources don't matter. Comp

I disagree strongly with this view that hard problems need to be "conceptually solved" in a vacuum first. That experimentation and resources don't matter. Computing increases have enabled a ton more experimentation with AI. Experiments that would have taken years to do, can be done in hours now. Experimentation lets scientists develop intuition about their algorithms. And get a sense of what will and won't work. And have a better mental model of the problem. Not to mention the benefit of having 10x or more funding and researchers working on it now that results have been shown. It's a common myth that there has been little or no innovation in NNs and that it's just a matter of computing power. Taking the best algorithms of the 90s and running them on modern hardware would still give you poor results. For instance, most of the 90s research was on very shallow nets, they didn't know how to properly train deep ones.