I think it was Nassim Taleb[1] that made a case for that engineering and tinkering drives science rather than the other way around, and that most of the knowled

I think it was Nassim Taleb[1] that made a case for that engineering and tinkering drives science rather than the other way around, and that most of the knowledge gained comes from the practitioners rather than the scientists, who swoop in to claim credit after the fact (he uses the jet-engine as an example of this). That's not to say fundamental science is unimportant, just that it's getting too much of the credit for all the stuff that's been built. There's a tremendous amount of know-how required to successfully build an airplane, a jet-engine, a processor, or to successfully train a large neural network. And while theoretical physics no doubt can explain it all, it's not sufficient to build something that works. I think the fact that we're seeing so much progress in deep learning without any theoretical basis is another example of this. [1] Antifragile, Chapter: History Written By the Losers